srch

README

srch

This is a filesystem indexing and search tool for your SuperTXT documents. Queries are feed in through standard input. Here is a simple example:

   srch <<EOF
   web browser
   EOF

The query syntax can be freeform text as shown above, which will query file content for matches. Queries can also be made against specific metadata fields. If there is a supertxt point like this:

   * bar: abc

That field can be queried directly as shown:

   srch <<EOF
   .bar:"abc"
   EOF

Date values with ISO-8601 date formats can be queried by comparison. Assuming that there is a supertxt point with a date value

   * foo: 2015-07-08T12:15:42Z

This can be found with the following query that finds any values earlier than January 1, 2016:

   srch <<EOF
   .foo:<"2016-01-01"
   EOF

Note: All files are indexed with the ".dateModified" field automatically based on the modification time reported by the filesystem.

Geographical coordinates of the form (lat, long) can be queried using a special query syntax.

   srch <<EOF
   within:"5km of 17.021111, 99.703611"
   EOF

The coordinates can be replaced with an arbitrary string that will be best effort geolocated:

   srch <<EOF
   within:"100m of Colchester, England"
   EOF

For more details see the query string documentation for the bleve library.

=> https://blevesearch.com/docs/Query-String-Query/

Site searches

Using the "-C" parameter, you can specify not only an alternate directory to index and search, but also a different site (ie. server). If srch is installed on the remote server and is available on the PATH then you can search it like this over SSH:

   srch -C myserver: <<EOF
   something of interest
   EOF

The colon after myserver is important since it distinguishes between a local folder called myserver and a host name. You can also specify a username in the usual SSH style used by other SSHLA tools like scp, and git.

   srch -C user1@myserver: <<EOF
   modest proposal
   EOF

Any text after the colon is interpreted as the destination directory to search. The default is whatever the current working directory (usually, your user's home) on the remote machine.

If the server doesn't work with SSH and it has its own query API, there is a srch "plugin" architecture that will delegate server specific querying to a program that can work with that server's API. You run srch against a server like this.

   srch -C server1.com: <<EOF
   my query goes here
   EOF

Srch will check for the presence of a program called "server1.com" on your path. If such a program exists then srch will delegate to it to perform the query and return results using a "srch" subcommand. There is a wikipedia-specific srch program included with srch called "wikipedia.org" that handles queries to wikipedia.org.

   srch -C wikipedia.org: <<EOF
   Firth Forth
   EOF

Including additional file extensions

When srch encounters a file that doesn't have the SuperTXT (.s.txt) file extension it will attempt to read it through a file extension request as a SuperTXT file using cats. Cats itself checks to see if there's a program on your path that will convert the original file extension to what you're requesting using the heuristic "to" to identify a program that can make the conversion with a simple pipe. If such a program exists that can convert from the original file extension "tostxt" then srch will use it through cats to get a representation that it can parse and index. This can be a way to index other sorts of files so that they can be searched using srch. Even a simple shell script with the conventional file name can work.

Note: cats includes a program called "tostxt" that can be symbolically linked to "jpgtostxt" to extract metadata from exif-based formats to SuperTXT. You can use it to indexing locations, and dates of your pictures.