my-log

README

my-log

my-log is a tool for generating and parsing log files for whatever you want. This is early in development. Check our Roadmap before for what's working.

I originally wrote DropLogger to serve this purpose. DropLogger was originally designed to work primarily with IFTTT and Dropbox. Due to IFTTT changing their service significantly since I originally signed up, I no longer use it. Even without IFTTT, DropLogger is a great tool.

So, why did I decide to completely rewrite it? Mainly because DropLogger is written in Python. While Python is a great language, I haven't used it seriously for many years, and I didn't find a whole lot of motivation to add new features to DropLogger, due to this. But I've been working in go for the past six months, and have kind of fell in love with the language. I'd been considering a rewrite of DropLogger for a while, so I decided to help myself get more practice in go by rewriting DropLogger in it.

So, how does this work?

Currently, it mostly doesn't. my-log is still in its early stages, and DropLogger is still needed for most of the functionality. So, how will it work?

Log files

We start with the individual log files. These were designed to be very flexible so that they could be written using a number of different tools. Originally, IFTTT recipes were created that would write to files in Dropbox, but this could be adapted to a number of other automation tools to automatically write as things happen.

What things? Well, maybe you use Tasker to trigger an action when you get home. You might want to keep a log of whenever you arrive at your house. Or, maybe you use Last.FM to keep track of your music listening habits, and you want to log whenever you listen to a music track. You could create a Zap in Zapier that responds to new scrobbles on Last.FM, and adds those scrobbles to a file.

Log format

As I mentioned, the format is intended to be very easy to write. Here's a sample:

@begin January 12, 2024 at 2:34PM - Title
@key value
@longKey this entry is long, and
spans multiple lines
@number 4
@bool true
@end

So, each entry starts with @begin and ends with @end. It must have a date and a title. It may also have additional data which is indicated by an @ at the beginning of the line. If I were to convert this to JSON (which my-log can do for you), it would look like:

{
    "title": "Title",
    "date": "2024-01-12T14:34:00Z",
    "key": "value",
    "longKey": "this entry is long, and\nspans multiple lines",
    "number": 4,
    "bool": true
}

A couple things to note:

• When outputting JSON, the date is converted to ISO-8601 format. The timezone used (if none was given in the original log) is your own local time.
• The newline in the longKey was preserved
• Different types are recognized and parsed correctly. It supports the following types:
  • string (default)
  • numbers
  • boolean values (true or false)
  • dates and times
  • A null value (the string "null", "nil", "none", or "~")
  • A raw JSON object/array

Since the extra fields are optional, the simplest log entry can be on a single line. For example, you might have a log file called notes.txt with this:

@begin February 3, 2015 at 01:33PM - Remember to call Mom @end
@begin February 4, 2015 at 07:45AM - Breakfast today was great! @end

As JSON, that would be:

[{
    "title":"Rember to call Mom",
    "date":"2015-02-03T13:33:00Z"
},{
    "title":"Breakfase today was great!",
    "date":"2015-02-04T07:45:00Z"
}]

Adding log entries

As was previously noted, the idea is that you can figure out the best way for you to add to the log file. But, my-log also comes with a command to add them from the command line. Run my-log drop --help for instructions on how to use it. But, here's a few examples:

my-log drop notes "Hello"
# Adds "@begin <date> - Hello @end" to notes.txt file

my-log drop -d "yesterday" calls "Talked with Jeremy" -f phone_number=+1-555-867-5309
# If today is January 2, 2024, adds the follow entry to calls.txt
# @begin January 1, 2024 at 12:00:00AM UTC - Talked with Jeremy
# @phone_number +1-555-867-5309 @end

my-log drop -d "1999-12-31T23:59:59Z" events "The end of the world" -f notes="As we know it" -j '{"artist":"R.E.M","slaps":true}'
# Adds the following entry to events.txt
# @begin December 31, 1999 at 11:59:59PM UTC - The end of the world
# @notes As we know it
# @artist R.E.M
# @slaps true @end

Output

This is a work in progress. More info coming soon. Short version is, we want to be able to output to multiple formats in multiple places.

Check DropLogger's documentation for info on how we want to make it work.

Configuration

We use a TOML file for configuration. The default location on Linux is ~/.config/my-log/config.toml. You can find the exact location by doing my-log -h and looking at the help for the --config flag. Running my-log config will save the default config file to the default location. The file is intended to be edited by hand. There is no mechanism within the program to modify the file, aside from saving the default one.

The default one has comments to help you out, but here's the options:

[input]

• path: The path to where the logs are located. This is usually ~/my-log, but if you want to store it in Dropbox, you might want it to be ~/Dropbox/my-log
• ext: The file extension for log files. This is usually txt, which makes it easier to work with multiple tools, but you can change it to log, or my-log, if you want. If you set it to an empty string, no extension will be used, which also means that when parsing the log files, it will look at all files in the folder.
• recurse: Whether to look in sub-folders.

[output.which-one]

Each separate output has its own set of configuration. So, replace which-one with the output name.

• enabled: if set to false, will skip that output when running.
• config: This is an output-specific set of settings

[output.stdout.config]

This section may change in the near future. We're considering supporting multiple formats.

• format: Which formatter to use when outputting data. This value is also used by my-log drop to output the new entry.

[formatters]

Some formatters may have custom configuration.

[formatters.json]

• pretty_print: If true, JSON output will be pretty printed. If false, it will be printed to a single line.

Roadmap

• drop command. This is functional, and supports all the features of drop-a-log
  • Don't add an extra blank line before new entries
  • Add a new line at the end
• Output log entries
  • A single date
  • a specific period of time
  • filter to specific logs
  • stdout
    • plain text
    • JSON
    • YAML
    • Other formats? Submit an issue!
  • file output
    • Any format that stdout supports
    • Multiple formats at once
    • RSS
    • ATOM
  • sqlite database
• Maybe: plug-in system to add formats or output destinations

Documentation

Overview

Copyright © 2024 Dan Jones <danjones@goodevilgenius.org>

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.