slackdump

README

Slack Dumper

Purpose: archive your private and public Slack messages, users, channels, files and emojis. Generate Slack Export without admin privileges.

Quick links:

• Join the discussion in Telegram.

• Buy me a cup of tea, or use Github Sponsors button on the top of the page.

• 

• How to's:

  • Mattermost migration steps
  • SlackLogViewerとSlackdumpを一緒に使用する
  • v1 Overview on Medium.com (outdated)

Description

Typical use scenarios:

• archive your private conversations from Slack when the administrator does not allow you to install applications OR you don't want to use potentially privacy-violating third-party tools,
• archive channels from Slack when you're on a free "no archive" subscription, so you don't lose valuable knowledge in those channels,
• create a Slack Export archive without admin access, or
• save your favourite emojis.

There are four modes of operation (more on this in User Guide):

1. List users/channels
2. Dumping messages and threads
3. Creating a Slack Export in Mattermost or Standard modes.
4. Emoji download mode.

Slackdump accepts two types of input (see Dumping Conversations section):

1. the URL/link of the channel or thread, OR
2. the ID of the channel.

Quick Start

On macOS, you can install Slackdump with Homebrew::

brew install slackdump

On other Operating Systems, please follow these steps:

1. Download the latest release for your operating system from the releases page.
2. Unpack the archive to any directory.
3. Run the ./slackdump or slackdump.exe executable (see note below).
4. You know the drill: use arrow keys to select the menu item, and Enter (or Return) to confirm.

By default, Slackdump uses the EZ-Login 3000 automatic login, and interactive mode.

[!NOTE] On Windows and macOS you may be presented with "Unknown developer" window, this is fine. Reason for this is that the executable hasn't been signed by the developer certificate.

To work around this:

• on Windows: click "more information", and press "Run Anyway" button.
• on macOS 14 Sonoma and prior: open the folder in Finder, hold Option and double click the executable, choose Run.
• on macOS 15 Sequoia and later: start the slackdump, OS will show the "Unknown developer" window, then go to System Preferences -> Security and Privacy -> General, and press "Open Anyway" button.

Slackord2: Migrating to Discord

If you're migrating to Discord, the recommended way is to use Slackord2 — a great tool with a nice GUI, that is compatible with the export files generated by Slackdump.

User Guide

For more advanced features and instructions, please see the User Guide.

Previewing Results

Once the workspace data is dumped, you can run built-in viewer:

slackdump view <zip or directory>

The built-in viewer supports all types of dumps:

1. Slackdump Archive format;
2. Standard and Mattermost Slack Export;
3. Dump mode files

The built-in viewer is experimental, any contributions to make it better looking are welcome.

Alternatively, you can use one of the following tools to preview the export results:

• SlackLogViewer - a fast and powerful Slack Export viewer written in C++, works on Export files (images won't be displayed, unless you used an export token flag).
• Slackdump2Html - a great Python application that converts Slack Dump to a static browsable HTML. It works on Dump mode files.
• slack export viewer - Slack Export Viewer is a well known viewer for slack export files.

Using as a library

Download:

go get github.com/rusq/slackdump/v3

Example

package main

import (
  "context"
  "log"

  "github.com/rusq/slackdump/v2"
  "github.com/rusq/slackdump/v2/auth"
)

func main() {
  provider, err := auth.NewValueAuth("xoxc-...", "xoxd-...")
  if err != nil {
      log.Print(err)
      return
  }
  sd, err := slackdump.New(context.Background(), provider)
  if err != nil {
      log.Print(err)
      return
  }
  _ = sd
}

See Package Documentation.

Using Custom Logger

Slackdump uses a "log/slog" package, it defaults to "slog.Default()". Set the default slog logger to the one you want to use.

If you were using logger.Silent before, you would need to implement a discarding Handler for slog.

## FAQ

#### Do I need to create a Slack application?

No, you don't.  Just run the application and EZ-Login 3000 will take
care of the authentication or, alternatively, grab that token and
cookie from the browser Slack session.  See [User's Guide][ug].



#### I'm getting "invalid_auth" error

Go get the new Cookie from the browser and Token as well.

#### Slackdump takes a very long time to cache users

Disable the user cache with `-no-user-cache` flag.

#### How to read the export file?

```shell
slackdump view <ZIP-archive or directory>

My Slack Workspace is on the Free plan. Can I get data older than 90-days?

No, unfortunately you can't. Slack doesn't allow to export data older than 90 days for free workspaces, the API does not return any data before 90 days for workspaces on the Free plan.

Thank you

Big thanks to all contributors, who submitted a pull request, reported a bug, suggested a feature, helped to reproduce, or spent time chatting with me on the Telegram or Slack to help to understand the issue and tested the proposed solution.

Also, I'd like to thank all those who made a donation to support the project:

• Vivek R.
• Fabian I.
• Ori P.
• Shir B. L.
• Emin G.
• Robert Z.
• Sudhanshu J.

Bulletin Board

Messages that were conveyed with the donations:

• 25/01/2022: Stay away from TheSignChef.com, ya hear, they don't pay what they owe to their employees.

Documentation

Index

• Variables
• type Option
  • func WithFilesystem(fs fsadapter.FS) Option
  • func WithForceEnterprise(b bool) Option
  • func WithLimits(l network.Limits) Option
  • func WithLogger(l *slog.Logger) Option
  • func WithSlackClient(cl clienter) Option
  • func WithUserCacheRetention(d time.Duration) Option
• type ProcessFunc
• type ProcessResult
  • func (pr ProcessResult) String() string
• type ProcessResults
  • func (prs ProcessResults) String() string
• type Session
  • func New(ctx context.Context, prov auth.Provider, opts ...Option) (*Session, error)
  • func NewNoValidate(ctx context.Context, prov auth.Provider, opts ...Option) (*Session, error)
  • func (s *Session) Client() *slack.Client
  • func (s *Session) CurrentUserID() string
  • func (s *Session) Dump(ctx context.Context, link string, oldest, latest time.Time, ...) (*types.Conversation, error)
  • func (s *Session) DumpAll(ctx context.Context, link string) (*types.Conversation, error)
  • func (s *Session) DumpEmojis(ctx context.Context) (map[string]string, error)
  • func (s *Session) DumpRaw(ctx context.Context, link string, oldest, latest time.Time, ...) (*types.Conversation, error)
  • func (s *Session) GetChannels(ctx context.Context, chanTypes ...string) (types.Channels, error)
  • func (s *Session) GetUsers(ctx context.Context) (types.Users, error)
  • func (s *Session) Info() *WorkspaceInfo
  • func (s *Session) Stream(opts ...stream.Option) *stream.Stream
  • func (s *Session) StreamChannels(ctx context.Context, chanTypes []string, cb func(ch slack.Channel) error) error
• type Slacker
• type WorkspaceInfo

Examples

• New (BrowserAuth)
• New (CookieFile)
• New (TokenAndCookie)

Variables

var AllChanTypes = []string{"mpim", "im", "public_channel", "private_channel"}

AllChanTypes enumerates all API-supported channel types as of 03/2023.

var ErrNoUserCache = errors.New("user cache unavailable")

ErrNoUserCache is returned when the user cache is not initialised.

Types

type Option

type Option func(*Session)

Option is the signature of the option-setting function.

func WithFilesystem

func WithFilesystem(fs fsadapter.FS) Option

WithFilesystem sets the filesystem adapter to use for the session. If this option is not given, the default filesystem adapter is initialised with the base location specified in the Config.

func WithForceEnterprise

func WithForceEnterprise(b bool) Option

func WithLimits

func WithLimits(l network.Limits) Option

WithLimits sets the API limits to use for the session. If this option is not given, the default limits are initialised with the values specified in DefLimits.

func WithLogger

func WithLogger(l *slog.Logger) Option

WithLogger sets the logger to use for the session. If this option is not given, the default logger is initialised with the filename specified in Config.Logfile. If the Config.Logfile is empty, the default logger writes to STDERR.

func WithSlackClient

func WithSlackClient(cl clienter) Option

WithSlackClient sets the Slack client to use for the session. If this

func WithUserCacheRetention

func WithUserCacheRetention(d time.Duration) Option

WithUserCacheRetention sets the retention period for the user cache. If this option is not given, the default value is 60 minutes.

type ProcessFunc

type ProcessFunc func(msg []types.Message, channelID string) (ProcessResult, error)

ProcessFunc is the signature of the processor function that Dump* family functions accept and call for each API call result. It can be used to modify in-place the slice of messages, returned from API, before they are appended to the slice that will be returned by Dump*. Messages passed to this function are unsorted.

type ProcessResult

type ProcessResult struct {
	// Entity is the type of entity that this result is related to.
	Entity string
	// Count is the number of items processed during processing.
	Count int
}

ProcessResult contains the result of processing.

func (ProcessResult) String

func (pr ProcessResult) String() string

type ProcessResults

type ProcessResults []ProcessResult

ProcessResults is the slice of ProcessResult

func (ProcessResults) String

func (prs ProcessResults) String() string

type Session

type Session struct {
	// contains filtered or unexported fields
}

Session stores basic session parameters. Zero value is not usable, must be initialised with New.

func New

func New(ctx context.Context, prov auth.Provider, opts ...Option) (*Session, error)

New creates new Slackdump session with provided options, and populates the internal cache of users and channels for lookups. If it fails to authenticate, AuthError is returned.

Example (BrowserAuth)

provider, err := auth.NewBrowserAuth(context.Background())
if err != nil {
	log.Print(err)
	return
}
fsa := openTempFS()
defer fsa.Close()
sd, err := New(context.Background(), provider)
if err != nil {
	log.Print(err)
	return
}
_ = sd

Example (CookieFile)

provider, err := auth.NewCookieFileAuth("xoxc-...", "cookies.txt")
if err != nil {
	log.Print(err)
	return
}
fsa := openTempFS()
defer fsa.Close()

sd, err := New(context.Background(), provider)
if err != nil {
	log.Print(err)
	return
}
_ = sd

Example (TokenAndCookie)

provider, err := auth.NewValueAuth("xoxc-...", "xoxd-...")
if err != nil {
	log.Print(err)
	return
}
fsa := openTempFS()
defer fsa.Close()

sd, err := New(context.Background(), provider)
if err != nil {
	log.Print(err)
	return
}
_ = sd

func NewNoValidate

func NewNoValidate(ctx context.Context, prov auth.Provider, opts ...Option) (*Session, error)

NewNoValidate creates new Slackdump session with provided options, and populates the internal cache of users and channels for lookups. This function does not validate the auth.Provider.

func (*Session) Client

func (s *Session) Client() *slack.Client

Client returns the underlying slack.Client.

func (*Session) CurrentUserID

func (s *Session) CurrentUserID() string

CurrentUserID returns the user ID of the authenticated user.

func (*Session) Dump

func (s *Session) Dump(ctx context.Context, link string, oldest, latest time.Time, processFn ...ProcessFunc) (*types.Conversation, error)

Dump dumps messages or threads specified by link. link can be one of the following:

• Channel URL - i.e. https://ora600.slack.com/archives/CHM82GF99
• Thread URL - i.e. https://ora600.slack.com/archives/CHM82GF99/p1577694990000400
• ChannelID - i.e. CHM82GF99
• ChannelID:ThreadTS - i.e. CHM82GF99:1577694990.000400

oldest and latest timestamps set a timeframe within which the messages should be retrieved, also one can provide process functions.

func (*Session) DumpAll

func (s *Session) DumpAll(ctx context.Context, link string) (*types.Conversation, error)

DumpAll dumps all messages. See description of Dump for what can be provided in link.

func (*Session) DumpEmojis

func (s *Session) DumpEmojis(ctx context.Context) (map[string]string, error)

func (*Session) DumpRaw

func (s *Session) DumpRaw(ctx context.Context, link string, oldest, latest time.Time, processFn ...ProcessFunc) (*types.Conversation, error)

DumpRaw dumps all messages, but does not account for any options defined, such as DumpFiles, instead, the caller must hassle about any processFns they want to apply.

func (*Session) GetChannels

func (s *Session) GetChannels(ctx context.Context, chanTypes ...string) (types.Channels, error)

GetChannels list all conversations for a user. `chanTypes` specifies the type of messages to fetch. See github.com/rusq/slack docs for possible values. If large number of channels is to be returned, consider using StreamChannels.

func (*Session) GetUsers

func (s *Session) GetUsers(ctx context.Context) (types.Users, error)

GetUsers retrieves all users either from cache or from the API. If Session.usercache is not empty, it will return the cached users. Otherwise, it will try fetching them from the API and cache them.

func (*Session) Info

func (s *Session) Info() *WorkspaceInfo

Info returns a workspace information. Slackdump retrieves workspace information during the initialisation when performing authentication test, so no API call is involved at this point.

func (*Session) Stream

func (s *Session) Stream(opts ...stream.Option) *stream.Stream

Stream streams the channel, calling proc functions for each chunk.

func (*Session) StreamChannels

func (s *Session) StreamChannels(ctx context.Context, chanTypes []string, cb func(ch slack.Channel) error) error

StreamChannels requests the channels from the API and calls the callback function cb for each.

type Slacker

type Slacker interface {
	AuthTestContext(context.Context) (response *slack.AuthTestResponse, err error)
	GetConversationHistoryContext(ctx context.Context, params *slack.GetConversationHistoryParameters) (*slack.GetConversationHistoryResponse, error)
	GetConversationRepliesContext(ctx context.Context, params *slack.GetConversationRepliesParameters) (msgs []slack.Message, hasMore bool, nextCursor string, err error)
	GetUsersPaginated(options ...slack.GetUsersOption) slack.UserPagination

	GetStarredContext(ctx context.Context, params slack.StarsParameters) ([]slack.StarredItem, *slack.Paging, error)
	ListBookmarks(channelID string) ([]slack.Bookmark, error)

	GetConversationsContext(ctx context.Context, params *slack.GetConversationsParameters) (channels []slack.Channel, nextCursor string, err error)
	GetConversationInfoContext(ctx context.Context, input *slack.GetConversationInfoInput) (*slack.Channel, error)
	GetUsersInConversationContext(ctx context.Context, params *slack.GetUsersInConversationParameters) ([]string, string, error)

	SearchMessagesContext(ctx context.Context, query string, params slack.SearchParameters) (*slack.SearchMessages, error)
	SearchFilesContext(ctx context.Context, query string, params slack.SearchParameters) (*slack.SearchFiles, error)
}

Slacker is the interface with some functions of slack.Client.

type WorkspaceInfo

type WorkspaceInfo = slack.AuthTestResponse

WorkspaceInfo is an type alias for slack.AuthTestResponse.