burl

README

#+PROPERTY: header-args :eval never-export :exports code :results silent
#+TITLE: bURL — Native Messaging Application for LinkRemark

This is a helper application for the LinkRemark browser extension
(install it [[https://addons.mozilla.org/en-GB/firefox/addon/linkremark/][to Firefox from addons.mozilla.org]],
visit [[https://github.com/maxnikulin/linkremark/][project page at GitHub]])
which goal is more convenient note taking
on visited web pages using [[https://orgmode.org][Org Mode]] for [[https://www.gnu.org/software/emacs/][GNU Emacs]].

#+ATTR_HTML: :alt Screenshot of Preview & Debug Info page of LinkRemark extension for capture of https://orgmode.org/ when org-manual.org and org-guide.org are configured as note files for bURL
#+attr_html: :style max-height: 50%
[[file:./burl-linkremark-preview-demo.png]]

# Sorry, Org renderer on GitHub could not transform info: to https: links
Such kind of application is called native messaging host
(application, backend) for a browser add-on. It serves as a bridge
that allows to run [[https://www.gnu.org/software/emacs/manual/html_node/emacs/Invoking-emacsclient.html][emacsclient]]
([[info:emacs#Invoking emacsclient][info "(emacs) Invoking emacsclient"]])
and to search URLs in your notes.
Browser extension can not do it directly due to security reasons.
In respect to capturing notes, bURL can work with
[[https://orgmode.org/manual/Protocols.html][org-protocol]]
([[info:org#Protocols][info "(org) Protocols"]])
using [[https://www.gnu.org/software/emacs/manual/html_node/emacs/Invoking-emacsclient.html][emacsclient]]
directly, so it is not necessary
[[https://orgmode.org/worg/org-contrib/org-protocol.html][to setup desktop-wide org-protocol: handler]].
LinkRemark source code repository has a simple native messaging app,
[[https://github.com/maxnikulin/linkremark/blob/master/examples/backend-python/lr_emacsclient.py][lr_emacsclient.py Python script]],
that calls emacsclient for org-protocol: URIs representing captures.
The advantage of bURL is that
with a bit more configuration, browser add-on will show
headings with the same URL as the captured page (a frame on it, or the link)
and will allow to open a file in Emacs at particular location.
Both features are optional.
You are free to capture notes using global
[[https://orgmode.org/manual/Protocols.html][org-protocol]]
handler or even to copy notes to clipboard.
URL lookup works only if a list of files to search is specified.

*Warning* There is a problem with browsers distributed
as snap or flatpack. Such applications works with
additional level of isolation in respect to system files,
so external application can not be invoked even through
native messaging API. Chromium in Ubuntu is shipped as
snap package only since Ubuntu-20.04 LTS focal, for Firefox snap is made
default option in Ubuntu-21.10 impish, but Firefox is still
can be installed as a deb package using apt.
Related Firefox bugs:
- [[https://bugzilla.mozilla.org/show_bug.cgi?id=1661935][Bug 1661935: Snap: cannot install/manage extensions from extensions.gnome.org]],
- [[https://bugzilla.mozilla.org/show_bug.cgi?id=1621763][Bug 1621763: (flatpak) native messaging support missing]].

The helper is tested only on Linux. The stage of its development
currently is "proof of concept": wrapper shell script serves
as a configuration file, only exact matches of URLs are checked.
URL normalization e.g. default index page,
stripping of user activity tracking query parameters are only planned.

* Install

# src_elisp{} is rendered literally by GitHub,
# so elisp: links are used.
# There are zero width spaces between link brackets
# and "=" verbatim markers to test whether it may be
# a workaround against markers rendered inside link description.
To capture pages using bURL,
[[https://www.gnu.org/software/emacs/manual/html_node/emacs/Emacs-Server.html][server]]
([[info:emacs#Emacs Server][info "(emacs) Emacs Server"]]),
should be running in Emacs: [[elisp:(server-start)][M-x server-start]] to try it,
~(server-start)~ in config file, or start Emacs in daemon mode
(maybe as a user systemd service) as a long term solution.
Without Emacs server the backend is still able to check
presence of URLs in configured files, but links to particular lines
from LinkRemark preview page do not work.

It is necessary to load [[https://orgmode.org/manual/Protocols.html][org-protocol]]
([[info:org#Protocols][info "(org) Protocols"]])
e.g. by customizing list of Org packages
[[elisp:(customize-variable-other-window 'org-modules)][M-: (customize-variable-other-window 'org-modules)]]
unless you are going to insert captured notes using
[[help:org-paste-subtree][C-c C-x C-y, org-paste-subtree]].

Certainly you need to download the binary or to build it yourself from sources.
GitHub project page has the [[https://github.com/maxnikulin/burl/releases/][releases]] section.
Programming language for this project is Go. To compile it should be enough
to type
#+begin_src bash
  make
#+end_src
and =burl_backend= file will be created.

In a minimal variant you need just to create a native messaging manifest
file. Browsers consider it as a permission for a particular add-on
to run the described external application. See the following references
for more detailed description and precise location of manifest files:

- Firefox: <https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_manifests>,
  <https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_messaging>
- Chrome: <https://developer.chrome.com/apps/nativeMessaging>

You may create manifest file using any text editor (e.g. Emacs)
or use a special command line option:

#+begin_src bash
  : "$(moznathostsdir:="$HOME/.mozilla/native-messaging-hosts/")"
  test -d "$moznathostsdir" || mkdir "$moznathostsdir"
  "${bindir:-.}/burl_backend" -manifest-firefox "$moznathostsdir"
#+end_src

It creates a JSON file with the name of the application, path to executable file,
and list of extensions that are allowed to run this application.
Next open LinkRemark configuration in browser, grant permission for native
messaging (requested during installing in Firefox while compatibility with
version 78 ESR is maintained) and specify backend name, that is "=io.github.maxnikulin.burl="
by default. Google Chrome and Chromium browsers have different search
paths for native manifest and another field to specify list
of allowed extensions (so =-manifest-chrome= should be passed).

You may specify list of files to search whether some URL has been captured already.
There is no a dedicated configuration file yet, however
the same effect could be achieved by creating of a small shell
wrapper that passes list of files to =burl_backend= binary.
Manifest should contain path to the wrapper.
It is possible to create the wrapper and the manifest at once.
Likely you would like to adjust list of files in the following example:

#+begin_src bash :var orgdir=(eval org-directory)
  : "$(orgdir:="$HOME/org")"
  : "$(moznathostsdir:="$HOME/.mozilla/native-messaging-hosts/")"
  test -d "$moznathostsdir" || mkdir "$moznathostsdir"
  "${bindir:-.}/burl_backend" -manifest-firefox "$moznathostsdir" \
      -wrapper ~/.local/bin/burl_wrapper \
      -org "${orgdir}/notes.org" \
      -org "${orgdir}/capture.org"
#+end_src

Pass =-backend NAME= option to use custom native host name instead of
default =io.github.maxnikulin.burl=, e.g. =burl_wrapper=.
There is no requirement that executable should be in your PATH,
so some custom directory (XDG does not specify =libexecdir= for users)
may be better than =~/.local/bin=.

The result should be like the following:

# The folloing is not supported by GitHub renderer
#    #+caption: [[file:~/.mozilla/native-messaging-hosts/io.github.maxnikulin.burl.json][=~/.mozilla/native-messaging-hosts/io.github.maxnikulin.burl.json=]]
[[file:~/.mozilla/native-messaging-hosts/io.github.maxnikulin.burl.json][~/.mozilla/native-messaging-hosts/io.github.maxnikulin.burl.json]]
#+begin_example
  {
    "name": "io.github.maxnikulin.burl",
    "description": "Burl - LinkRemark interface to Emacs",
    "path": "/home/ubuntu/.local/bin/burl_wrapper",
    "type": "stdio",
    "allowed_extensions": [
      "linkremark@maxnikulin.github.io"
    ]
  }
#+end_example

Omitting error-handling stuff, wrapper looks like

# Unsupported by GitHub: #+caption:
[[file:~/.local/bin/burl_wrapper][~/.local/bin/burl_wrapper]]
#+begin_example
  #!/bin/sh -eu
  exec /home/ubuntu/.local/bin/burl_backend --log - \
    --org /home/ubuntu/org/notes.org \
    --org /home/ubuntu/org/capture.org
#+end_example

Open LinkRemark settings from context menu for the extension toolbar button
or from the =about:addons= page.
- Set "native-messaging" for export method in "Communication Channel" section
  to pass captures to Emacs using bURL (optional).
- Set name of native messaging backend to "io.github.maxnikulin.burl" (default)
  or to the name you specified in the native messaging manifest.
- Grant native messaging permission (in Firefox it is requested during
  installation to keep compatibility with Firefox-78 ESR that does not
  allow it as an optional permission).

If you have provided list of files, you can try URL lookup
form the add-on preview & debug info page. The "Mentions" sections
allows to check any URLs (one per line).

* Tuning of Emacs
  :PROPERTIES:
  :CUSTOM_ID: emacs-tune
  :END:

** Open Org file at particular line
   :PROPERTIES:
   :CUSTOM_ID: emacs-open-at-line
   :END:

When a file is opened at particular line by clicking on some link
inside the mentions section of the add-on preview page, it may happen
that some parent headings are collapsed. I have not realized
what is the safe way to unfold necessary outline headings without
possible destructive interference with other invocations of =emacsclient=.

For a while you may alleviate the issue by adding the following peace
of code to your Emacs init file

#+begin_src elisp
  (defun lr-org-reveal-folded-line ()
    (when (derived-mode-p 'org-mode)
      (org-reveal)))

  (add-hook 'server-visit-hook #'lr-org-reveal-folded-line)
#+end_src

Alternatively use =C-c C-r= that is a binding for =M-x= [[help:org-reveal][org-reveal]].

** Frame (window) for capture or to show existing link
   :PROPERTIES:
   :CUSTOM_ID: emacs-ensure-frame
   :END:

I am unsure what is the best way to create frame (if it does not exist yet)
for capture or to show the place where an URL is mentioned.
Personally I do not like when Emacs creates new frame
in a terminal window just because I have launched a browser
from this terminal. So ~linkremark-ensure-frame~ function
is invoked (if it is defined) before over actions. An example
how to it may look like:

#+begin_src elisp
  (defun linkremark-ensure-frame ()
    (or (memq 'x (mapcar #'framep (frame-list)))
	(select-frame
	 (make-frame '((name . "LinkRemark") (window-system . x))))))
#+end_src

It may open a file in a frame located on another desktop.
I have not figured out how to cook something better from the following
ingredients: ~server-window~, ~display-buffer-alist~,
~frames-on-display-list~, ~default-frame-alist~.

* WebExtensions package for Go

This project has a simple Go package that provides tools for using
[[https://pkg.go.dev/net/rpc/jsonrpc][net/rpc/jsonrpc]] (frozen) package from Go standard library through =stdin=
and =stdout= pipes accordingly to native messaging protocol in browsers.
Its advantage is no external dependencies, its disadvantages
are absence of support of request contexts and single parameter
of RPC methods that must be passed as single-element array.
You may consider JSON-RPC 1.0 as not flexible enough in comparison to
version 2.0 of the protocol.

See [[file:pkg/webextensions][pkg/webextensions]] folder for more detailed description.
The [[file:examples][examples]] directory contains a small backend and a Firefox
extension that can call its methods.

* License

GPL v3

An example of browser add-on [[file:examples/webextensions_addon][examples/webextensions_addon]] is licensed
under MPL-2.0 (it is no more than a demo for the [[file:pkg/webextensions][pkg/webextensions]] Go package).

# LocalWords: LinkRemark backend URIs bURL WebExtensions JSON-RPC
# LocalWords: GPL MPL flatpack stdin stdout systemd JSON RPC XDG