Permalink 04:46:06 pm, by mholmes, 119 words, 23 views   English (CA)
Categories: Activity log; Mins. worked: 360

Graves: Google search done; Solr search page coming along

Over the weekend I set up a Google Custom Search for Graves, and integrated it into the site. I did some final bugfixing this morning and that now seems to be working.

Today I got back into working on the Solr search, and after a quick addition to the Solr index (which will have to be propagated to the Library's Solr servers if they agree to host the index), I've built a large "Advanced search" page which has lots of options for filtering by named entities and so on. I've got the form completed, and I've started on the JS class which will handle the actual search. I'm developing against my local install of Solr, which is working well.


Permalink 03:59:35 pm, by mholmes, 77 words, 19 views   English (CA)
Categories: Activity log; Mins. worked: 300

Excellent progress with Solr

After consulting with BJ, I've now got the Graves build process creating a collection of XML files optimized for Solr indexing with the default schema, and I've tested them with a local Solr instance. I'm confident I can build a nicely-faceted search page which uses a remote Solr backend to support rich queries with syntax highlighting. This was a little easier than I thought it was going to be. I think I like Solr quite a lot.


Permalink 04:40:59 pm, by mholmes, 113 words, 17 views   English (CA)
Categories: Activity log; Mins. worked: 180

Graves search: on to Solr and Google

I now have the Graves site creating a Google-style sitemap, which is referenced in robots.txt. That's a prerequisite for setting up a Google search page, which should be fairly straightforward. I've also got a working Solr implementation on my computer and I've been testing and learning about it once again. I think the logical approach is going to be to create JSON for ingestion into Solr, which will give the ultimate in flexibility and enable us to create a faceted search interface. The Solr examples have a single file for each collection, but I think we're going to want to break it up; I'm not quite sure how to handle that yet.


Permalink 05:02:18 pm, by mholmes, 85 words, 16 views   English (CA)
Categories: Activity log; Mins. worked: 120

Completed the Graves integration of local search

A number of additional tweaks and feature updates to complete the local search part of this project:

  • Search functionality tweaked to strip out accented characters in the search tokens.
  • Search results now include a list of the tokens actually searched (not stopped or too short)
  • Exist can now supply the simple local search at the search.html URL.
  • Some nifty flex tweaks make the wrapping of the home page more friendly to keeping the search content together with its results.
  • Everything is tested under https.


Permalink 03:35:54 pm, by mholmes, 125 words, 15 views   English (CA)
Categories: Activity log; Mins. worked: 300

Graves: local search now working, with eXist override

The local search functionality is now done, and it also incorporates document type and date range filtering. I've added an override to the eXist instance such that the eXist search page is shown when it's running on eXist, and that works fine too. The JSON files are 58MB, which is not a significant extra load. The only slight wrinkle I might go back and address is the handling of accented characters; there are some (because of Spanish etc.), but the Porter stemmer is not expecting any, so they seem to have their accents stripped. I've removed the accented character entry buttons from the local search page in view of this, but they're easy to put back if we figure out how not to lose them.


Permalink 04:50:36 pm, by mholmes, 67 words, 17 views   English (CA)
Categories: Activity log; Mins. worked: 120

Graves: implementing local JSON search

Ported the code over from the Keats site to generate the search JSON. The Graves site has about the same number of terms, but appears to generate an index only about two-thirds of the size of the Keats. I have everything working in the build process, but I still have to figure out what happens on the front-end to distinguish between the local and the eXist-based search.


Permalink 04:02:52 pm, by mholmes, 56 words, 22 views   English (CA)
Categories: Activity log; Mins. worked: 30

Scare over OAC site

The OAC folks wrote to say that they had deleted our site without backup (and apparently without warning); this turned out not to be the case, but just in case I went into the admin interface to get myself a backup, and discovered that's not an option. So I've curled the whole site just in case.


Permalink 04:01:07 pm, by mholmes, 25 words, 22 views   English (CA)
Categories: Activity log; Mins. worked: 15

EC now encoding interviews

Got EC set up with endings SVN project for encoding of interview transcriptions. The schema and encoding strategies seem to be working well so far.


Permalink 03:27:51 pm, by mholmes, 27 words, 21 views   English (CA)
Categories: Activity log; Mins. worked: 150

Interview #2 and project meeting

With MH, LG and CC, did the second interview, which went well; a good hour of material. Then had small project meeting (minutes to go on GitHub).


Permalink 04:15:48 pm, by mholmes, 42 words, 19 views   English (CA)
Categories: Activity log; Mins. worked: 120

More work on search

The search now more effectively sorts hits in order of relevance, prioritizing those which contain more of the search terms, and within those sets sorting by total number of instances. Tweaked the styling a bit too, and fixed a couple of oddities.

<< Previous Page :: Next Page >>


Blog for the SSHRC-funded Endings project, 2016-2020


XML Feeds