geneos

README

geneos management tool

The geneos program will help you manage your Geneos environment on Linux.

Aims

• Make your life easier; at least the part managing Geneos
• Keep it simple through the Principle of least astonishment
• Help you use automation tools with Geneos

Basic Features

• Initialise a new installation with one command
• Adopt an existing installation that uses older tools
• Manage a group of servers with a single command
• Create and manage Geneos compatible AES256 encoded passwords
• Manage certificates for TLS connectivity between Geneos components
• Configure instance settings without editing files
• Download and install Geneos software, update components
• Simple bootstrapping of Self-Announcing Netprobes

Note

Please note that this README.md file and the pre-generated documentation it references are not always fully up-to-date and there will be small, possibly annoying or confusing, differences between functionality and this documentation. The command help text listing available flags is however generated from the code itself and should be generally up-to-date, except in those cases where internal behaviour doesn't match the flag - once again apologies. As this is an OSS project contributions are more than welcome, but will probably involve reading code rather than correcting grammar and spelling.

Getting Started

The geneos program breaks down functionality into subsystems and commands. Each of the subsystems has it's own set of commands. The subsystems available are:

• geneos aes - AES256 Encryption and decryption commands using keyfiles
• geneos config - Configuration options for the geneos program itself
• geneos host - Remote host configuration commands
• geneos init - Environment initialisation commands
• geneos package - Geneos package management
• geneos tls - TLS/SSL certificate management

If you use any of these named subsystems as commands then they will tend to do something useful, such as list available resources. Any of the instructions below that apply to a normal command apply to the commands inside each subsystem, e.g. geneos aes ls is an ls command for aes.

For typical commands the command line will be like this:

geneos COMMAND [flags] [TYPE] [NAMES...]

• COMMAND is the name of the command
• flags are dash or double-dash prefixed options. Many of these options will require arguments and the details for each command will be in the help test.
• TYPE is the optional component type
• NAMES is an optional list of instance names. They can have an optional @HOST suffix which restricts the match to only the configured HOST.

Some commands have special options which are documented in the individual pages linked below.

The overall goal when choosing command names and their options has been to make things more intuitive rather than less. This is not always possible and so some options will be different between commands while performing the same task; One examples is the -f (follow) flags for logs which has similar functionality to the -l (logs) flag for init commands.

Legacy Command Emulation

If you run the program with a name ending in ctl, either through a symbolic link or by copying the binary, then the legacy command syntax is emulated in a simplistic way. This will allow for users or automation scripts to continue working in their environment and get used to the geneos command syntax gradually. The first half of the executable name is mapped to the component type, so for example:

ln -s geneos gatewayctl
# this then runs ./geneos start gateway GW1
./gatewayctl GW1 start

ln -s geneos netprobectl
# this then runs ./geneos list netprobe
./netprobectl list

In general TYPEctl NAME COMMAND becomes geneos COMMAND TYPE NAME

The word all instead of a specific instance name is supported as expected.

Available Commands

Commands have their own automatically generated documents and are listed here: geneos.md

Concepts & Terminology

This documentation and the program itself assumes familiarity with the Geneos suite of products. Many of the key terms have been inherited from earlier systems.

The specific types supported by this program are details in Component Types.

Geneos

Geneos is a suite of software products from ITRS that provide real-time visibility of I.T. infrastructure and trading environments. It uses a three-tier architecture to collect, process and present enriched data to administrators.

Components

A component is a type of software package and associated data. Each component will typically be a software package from one of the three-tiers mentioned above but can also be a derivative, e.g. a Self-Announcing Netprobe is a component type that abstracts the special configuration of either a vanilla Netprobe or, for example, the Fix Analyser Netprobe.

The supported component types are listed in Component Types.

Instances

An instance is an independent copy of a component with a working directory (<top-level directory>/<component>/<component>s/<instance name>, e.g. /opt/itrs/netprobe/netprobes/myNetprobe), configuration and other persistent files. Instances share read-only package directories for the binaries and other files from the distribution for the specific version being used.

Hosts

Hosts are the locations that components are installed and instantiated. There is always a localhost.

Instance Protection

Individual instances can be protected again being stopped or deleted by setting it to be protected.

geneos protect gateway IMPORTANT_GW

This also applies to almost any command that stops an instance, such as the more obvious ones like restart but also disable and others. The copy command, because it must be given the name for a source, does not check this setting. For most commands that do check the protection setting before running there is a --force flag to override the protection. The delete command already requires that an instance be disabled or called with the --force flag.

If you run geneos delete host HOSTNAME with the --stop flag to stop instance on the remote host first, then the protected settings is also checked and the command will terminate on the first error. This does however mean that unprotected instances on that host may have been stopped in the meantime.

The update command will not run if any protected instance is using the base symlink about to be updated.

Environment Settings

The geneos program uses the packages Cobra and Viper (the latter via a wrapper package) to provide the command syntax and configuration management. There is full support for Viper's layered configuration for non-instance settings, which means you can override global and user settings with environment variables prefixed ITRS_, e.g. ITRS_DOWNLOAD_USERNAME overrides download.username

Instance Settings

Each instance has a configuration file. This is the most basic expression of an instance. New instances that you create will have a configuration file named after the component type plus the extension .json. Older instances which you have adopted from older control scripts will have a configuration file with the extension .rc

Legacy Configuration Files

Historical (aka. legacy) .rc files have a simple format of the form

GatePort=1234
GateUser=geneos

Where the prefix (Gate) also encodes the component type and the suffix (e.g. Port) is the setting. Any lines that do not contain the prefix are treated as environment variables and are evaluated and passed to the program on start. Lines that contain environment variables like ${HOME} will be expanded at run time. If the configuration is migrated, either through an explicit geneos migrate command or if a setting is changes through geneos set or similar then the value of the environment variable will be carried over and continue to be expanded at run-time. The geneos show command can be passed a --raw flag to show the unexpanded values, if any.

While the geneos program can parse and understand the legacy .rc files above it will never update them, instead migrating them to their .json equivalents either when required or when explicitly told to using the migrate command.

JSON Configuration Files

The .json configuration files share common parameters as well as component type specific settings. For brevity some of these parameters are overloaded and have different meanings depending on the component type they apply to.

While editing the configuration files directly is possible, it is best to use the set and unset commands to ensure the syntax is correct.

Special parameters

All instance types support custom environment variables being set or unset. This is done through the set and unset commands below, alongside the standard configuration parameters for each instance type.

Some component types, namely Gateways and SANs, support other special parameters via other command line flags. See the help text or the full documentation for the set and unset commands for more details.

To set an environment variable use this syntax:

geneos set netprobe example1 -e PATH_TO_SOMETHING=/file/path

If an entry already exists it is overwritten.

To remove an entry, use unset, like this

geneos unset netprobe example1 -e PATH_TO_SOMETHING

You can specify multiple entries by using the flag more than once:

geneos set netprobe example1 -e JAVA_HOME=/path -e ORACLE_HOME=/path2

Finally, if your environment variable value contains spaces then use quotes as appropriate to your shell to prevent those spaces being processed. In bash you can do any of these to achieve the same result:

geneos set netprobe example1 -e MYVAR="a string with spaces"
geneos set netprobe example1 -e "MYVAR=a string with spaces"

You can review the environment for any instance using the show command:

geneos show netprobe example1

Also. output is available from the command command to show what would be run when calling the start command:

geneos command netprobe example1

General Command Flags & Arguments

geneos COMMAND [flags] [TYPE] [NAME...] [PARAM...]

Where:

• COMMAND - one of the configured commands
• flags - Both general and command specific flags
• TYPE - the component type
• NAME - one or more instance names, optionally including the remote server
• PARAM - anything that isn't one of the above

In general, with the exception of COMMAND and TYPE, all parameters can be in any order as they are filtered into their types for most commands. Some commands require arguments in an exact order.

As an example, these have the same meaning:

geneos ls -c gateway one two three
geneos ls gateway one -c two three

Reserved instance names are case-insensitive. So, for example, "gateway", "Gateway" and "GATEWAY" are all reserved.

The NAME is of the format INSTANCE@REMOTE where either is optional. In general commands will wildcard the part not provided. There are special REMOTE names @localhost and @all - the former is, as the name suggests, the local server and @all is the same as not providing a remote name.

There is a special format for adding SANs in the form TYPE:NAME@REMOTE where TYPE can be used to select the underlying Netprobe type. This format is still accepted for all other commands but the TYPE is silently ignored.

File and URLs

In general all source file references support URLs, e.g. importing certificate and keys, license files, etc.

The primary exception is for Gateway include files used in templated configurations. If these are given as URLs then they are used in the configuration as URLs.

Configuration Files

General Configuration

• /etc/geneos/geneos.json - Global options
• ${HOME}/.config/geneos/geneos.json - User options
• Environment variables ITRS_option - where . is replaced by _, e.g. ITRS_DOWNLOAD_USERNAME

General options are loaded from the global config file first, then the user one and any environment variables override both files. The current options are:

• geneos

The home directory for all other commands. See Directory Layout below. If set the environment variable ITRS_HOME overrides any settings in the files. This is to maintain backward compatibility with older tools. The default, if not set anywhere else, is the home directory of the user running the command or, if running as root, the home directory of the geneos or itrs users (in that order). (To be fully implemented) This value is also set by the environment variables ITRS_HOME or ITRS_GENEOS

• download.url

The base URL for downloads for automating installations. Not yet used. If files are locally downloaded then this can either be a file:// style URL or a directory path.

• download.username download.password

  These specify the username and password to use when downloading packages. They can also be set as the environment variables, but the environment variables are not subject to expansion and so cannot contain Geneos encoded passwords (see below):

  • ITRS_DOWNLOAD_USERNAME
  • ITRS_DOWNLOAD_PASSWORD

• snapshot.username snapshot.password

  Similarly to the above, these specify the username and password to use when taking dataview snapshots. They can also be set as the environment variables, with the same restrictions as above:

  • ITRS_SNAPSHOT_USERNAME
  • ITRS_SNAPSHOT_PASSWORD

• GatewayPortRange & NetprobePortRange & LicdPortRange

...

Component Configuration

For compatibility with earlier tools, the per-component configurations are loaded from .rc files in the working directory of each component. The configuration names are also based on the original names, hence they can be obscure. the migrate command allows for the conversion of the .rc file to a JSON format one, the original .rc file being renamed to end .rc.orig and allowing the revert command to restore the original (without subsequent changes).

If you want to change settings you should first migrate the configuration and then use set to make changes.

Note that execution mode (e.g. GateMode) is not supported and all components run in the background.

geneos Configuration File

This configuration file - in JSON format - should be found in the home directory of the user as ~/.config/geneos/geneos.json.

Structure of the default file is as follows.

{
  "download": {
    "url": "https://resources.itrsgroup.com/download/latest/"
  },
  "fa2cleanlist": "*.old",
  "fa2portrange": "7030,7100-",
  "fa2purgelist": "fa2.log:fa2.txt:*.snooze:*.user_assignment",
  "facleanlist": "*.old",
  "faportrange": "7030,7100-",
  "fapurgelist": "fileagent.log:fileagent.txt",
  "gatewaycleanlist": "*.old:*.history",
  "gatewayportrange": "7039,7100-",
  "gatewaypurgelist": "gateway.log:gateway.txt:gateway.snooze:gateway.user_assignment:licences.cache:cache/:database/",
  "geneos": "/opt/itrs",
  "licdcleanlist": "*.old",
  "licdportrange": "7041,7100-",
  "licdpurgelist": "licd.log:licd.txt",
  "netprobecleanlist": "*.old",
  "netprobeportrange": "7036,7100-",
  "netprobepurgelist": "netprobe.log:netprobe.txt:*.snooze:*.user_assignment",
  "privatekeys": "id_rsa,id_ecdsa,id_ecdsa_sk,id_ed25519,id_ed25519_sk,id_dsa",
  "reservednames": "",
  "sancleanlist": "*.old",
  "sanportrange": "7036,7100-",
  "sanpurgelist": "san.log:san.txt:*.snooze:*.user_assignment",
  "webservercleanlist": "*.old",
  "webserverportrange": "8080,8100-",
  "webserverpurgelist": "logs/*.log:webserver.txt"
}

Note: This file should not require any changes, except for fields ending in portrange which may need to be adjusted based on your environment.

Host Configuration File

This configuration file - in JSON format - should be found in the home directory of the user as ~/.config/geneos/hosts.json.

Structure of the default file is as follows.

{
  "hosts": {
    "psapac-dev-02": {
      "geneos": "/opt/itrs",
      "hostname": "172.123.456.789",
      "name": "PsApac-Dev-02",
      "osinfo": {
        "ansi_color": "0;31",
        "bug_report_url": "https://bugzilla.redhat.com/",
        "cpe_name": "cpe:/o:redhat:enterprise_linux:7.7:GA:server",
        "home_url": "https://www.redhat.com/",
        "id": "rhel",
        "id_like": "fedora",
        "name": "Red Hat Enterprise Linux Server",
        "pretty_name": "Red Hat Enterprise Linux Server 7.7 (Maipo)",
        "redhat_bugzilla_product": "Red Hat Enterprise Linux 7",
        "redhat_bugzilla_product_version": "7.7",
        "redhat_support_product": "Red Hat Enterprise Linux",
        "redhat_support_product_version": "7.7",
        "variant": "Server",
        "variant_id": "server",
        "version": "7.7 (Maipo)",
        "version_id": "7.7"
      },
      "port": 22,
      "username": "itrs"
    }
  }
}

Instance Configuration File

These configuration files - in JSON format - should be found in sub-directories under the geneos base directory (typiocally /opt/itrs, /opt/itrs/geneos or /opt/geneos) as GENEOS_BASE_DIRECTORY/TYPE/TYPEs/INSTANCE/TYPE.json where:

• GENEOS_BASE_DIRECTORY is the base directory for geneos.
• TYPE is the component type (licd, gateway, netprobe, san, fa2, fileagent or webservcer).
• TYPEs is the component type followed by the letter "s" (lowercase) to indicate a plural.
• INSTANCE is the instance name.
• TYPE.json is a the file name (e.g. licd.json, gateway.json, etc.).]

Directory Layout

The geneos configuration setting or the environment variable ITRS_HOME points to the base directory for all subsequent operations. The layout follows that of the original gatewayctl etc.

Directory structure / hierarchy / layout is as follows:

/opt/itrs
├── fa2
│   └── fa2s
├── fileagent
│   └── fileagents
├── gateway
│   ├── gateway_config
│   ├── gateways
│   │   └── [gateway instance name]
│   ├── gateway_shared
│   └── templates
│       ├── gateway-instance.setup.xml.gotmpl
│       └── gateway.setup.xml.gotmpl
├── hosts
├── licd
│   └── licds
│       └── [licd instance name]
├── netprobe
│   └── netprobes
│       └── [netprobe instance name]
├── packages
│   ├── downloads
│   │   ├── geneos-gateway-6.0.0-linux-x64.tar.gz
│   │   ├── geneos-licd-6.0.0-linux-x64.tar.gz
│   │   ├── geneos-netprobe-6.0.2-linux-x64.tar.gz
│   │   └── geneos-web-server-6.0.0-linux-x64.tar.gz
│   ├── fa2
│   ├── fileagent
│   ├── gateway
│   │   ├── 6.0.0
│   │   └── active_prod -> 6.0.0
│   ├── licd
│   │   ├── 6.0.0
│   │   └── active_prod -> 6.0.0
│   ├── netprobe
│   │   ├── 6.0.2
│   │   └── active_prod -> 6.0.2
│   └── webserver
│       ├── 6.0.0
│       └── active_prod -> 6.0.0
├── san
│   ├── sans
│   └── templates
│       └── netprobe.setup.xml.gotmpl
└── webserver
    └── webservers
        └── [webserver instance name]

where:

• fa2/ (Fix Analyser) contains settings & instance data related to the fa2 component type.

  • fa2/fa2s/ contains one sub-directory for each Fix Analyser instance named after the fa2 instance. These sub-directory will be used as working directories for the corresponding instances.

• fileagent/ (File Agent for Fix Analyser) contains settings & instance data related to the fileagent component type.

  • fileagent/fileagents/ contains one sub-directory for each File Agent instance named after the file agent instance. These sub-directory will be used as working directories for the corresponding instances.

• gateway/ contains settings & instance data related to the gateway component type.

  • gateway/gateway_config/ contains common Gateway configuration as include XML files.
  • gateway/gateways/ contains one sub-directory for each Gateway instance named after the gateway instance. These sub-directories will be used as working directories for the corresponding gateway instances.
  • gateway/gateway_shared/ contains shared Gateway data such as include XML files or scritped tools.
  • gateway/templates/ contains Gateway configuration templates in the form of Golang XML templates.

• hosts/ contains configurations for supporting control of Geneos component instances running on remote hosts.

• licd/ (License Daemon) contains settings & instance data related to the licd component type.

  • licd/licds/ contains one sub-directory for each licd instance named after the licd instance. This sub-directories will be used as working directories for the corresponding License Daemon (licd) instance.

• netprobe/ contains settings & instance data related to the netprobe component type.

  • netprobe/netprobes/ contains one sub-directory for each Netprobe instance named after the netprobe instance. These sub-directories will be used as working directories for the corresponding netprobe instances.

• packages/ contains the Geneos binaries / software packages installed.

  • packages/downloads/ contains files downloaded from the ITRS download portal, or the file repository used.
  • packages/fa2/ contains one sub-directory for each version of Fix Analyser installed, as well as symlinks (e.g. active_prod) pointing to the current default version. These sub-directory will contain the corresponding binaries.
  • packages/fileagent/ contains one sub-directory for each version of File Agent installed, as well as symlinks (e.g. active_prod) pointing to the current default version. These sub-directory will contain the corresponding binaries.
  • packages/gateway/ contains one sub-directory for each version of Gateway installed, as well as a symlinks (e.g. active_prod) pointing to the current default version. These sub-directory will contain the corresponding binaries.
  • packages/licd/ contains one sub-directory for each version of License Daemon (licd) installed, as well as a symlinks (e.g. active_prod) pointing to the current default version. These sub-directory will contain the corresponding binaries.
  • packages/netprobe/ contains one sub-directory for each version of Netprobe installed, as well as a symlinks (e.g. active_prod) pointing to the current default version. These sub-directory will contain the corresponding binaries.
  • packages/webserver/ contains one sub-directory for each version of Webserver (for web dashboards) installed, as well as a symlinks (e.g. active_prod) pointing to the current default version. These sub-directory will contain the corresponding binaries.

• san/ (Self-Announcing Netprobe) contains settings & instance data related to the san component type.

  • san/sans/ contains one sub-directory for each Self-Announcing Netprobe instance named after the san instance. These sub-directories will be used as working directories for the corresponding san instances.
  • san/templates/ contains Self-Announcing Netprobe configuration templates in the form of Golang XML templates.

• webserver/ (Webserver for web dashbaords) contains settings & instance data related to the webserver component type.

  • webserver/webservers/ contains one sub-directory for each Webserver instance named after the webserver instance. These sub-directories will be used as working directories for the corresponding Webserver instances.

The bin/ directory and the default .rc files are ignored. Please be careful in case you have customised anything in bin/.

As a very quick recap, each component directory will have a subdirectory with the plural of the name (e.g. gateway/gateways) which will contain subdirectories, one per instance, and these act as the configuration and working directories for the individual processes. Taking an example gateway called Gateway1 the path will be: ${ITRS_HOME}/gateway/gateways/Gateway1.

This directory will be the working directory of the process and also contain an .rc configuration file - if using the legacy scripts (e.g. gatewayctl) - or a .json configuration file - if using the geneos utility - as well as a .txt file to capture the STDOUT and STDERR of the process.

There will also be an XML setup file and so on.

Installation

Download the binary

You can download a pre-built binary version (for Linux on amd64 only) from this link or like this:

curl -OL https://github.com/itrs-group/cordial/releases/latest/download/geneos
chmod 555 geneos
sudo mv geneos /usr/local/bin/

Build from source

To build from source you must have Go 1.20 or later installed:

One line installation

go install github.com/itrs-group/cordial/tools/geneos@latest

Make sure that the geneos program is in your normal PATH - or that $HOME/go/bin is if you used the method above - to make things simpler.

Download from github and build manually

Make sure you do not have an existing file or directory called geneos and then:

github clone https://github.com/itrs-group/cordial.git
cd geneos/cmd/geneos
go build
sudo mv geneos /usr/local/bin