README ¶
Discord Downloader
DOWNLOAD LATEST RELEASE BUILDS
This project is a fork of Seklfreak's discord-image-downloader-go
For list of differences and why I made an independent project, see below
This is a Discord bot program to download files posted in specified Discord channels to local folders. It can fetch highest possible quality files from various sources (listed below), aside from downloading any file directly linked or attached. See Features below for full list.
Need help? Have suggestions? Join the Discord server!
Sections
- Getting Running
- Getting Started
- Features
- Guide: Saving History (Cataloging Old Messages)
- Guide: Settings / Configuration
- List of Settings
WARNING: Discord does not allow Automated User Accounts (Self-Bots/User-Bots)
Read more in Discord Trust & Safety Team's Official Statement...
While this project works for user logins, I do not reccomend it as you risk account termination. If you can, use a proper Discord Bot user for this program.
NOTE: This only applies to real User Accounts, not Bot users. This program currently works for either.
Getting Running
Depending on your purpose for this program, there are various ways you can run it.
- Run the executable file for your platform. (Process managers like pm2 work well for this)
- Run automated image builds in Docker. (Google it).
- Mount your settings.json to
/root/settings.json
- Mount a folder named "database" to
/root/database
- Mount your save folders or the parent of your save folders within
/root/
- i.e.
X:\My Folder
to/root/My Folder
- i.e.
- Mount your settings.json to
- Install Golang and compile/run the source code yourself. (Google it)
Getting Started
You can either create a settings.json
following the examples & variables listed below, or have the program create a default file (if it is missing when you run the program, it will make one, and ask you if you want to enter in basic info for the new file).
- Ensure you follow proper JSON syntax to avoid any unexpected errors.
- Feeling lazy or having issues? Try this JSON Formatter/Validator.
Credentials...
- If using a Bot User, enter the token into the
"token"
setting. Remove the lines for"username"
and"password"
or leave blank (""
). To create a Bot User, go to discord.com/developers/applications and create aNew Application
. Once created, go toBot
and create. The token can be found on theBot
page. To invite to your server(s), go toOAuth2
and check"bot"
, copy the url, paste into browser and follow prompts for adding to server(s). - If using a Real User (Self-Bot), fill out the
"username"
and"password"
settings. Remove the line for"token"
or leave blank (""
). - If using a Real User (Self-Bot) with 2FA (Two-Factor Authentication), enter the token into the
"token"
setting. Remove the lines for"username"
and"password"
or leave blank (""
). Token can be found fromDeveloper Tools
in browser underlocalStorage.token
or in the Discord clientCtrl+Shift+I (Windows)
/Cmd+Option+I (Mac)
underApplication → Local Storage → https://discordapp.com → "token"
. You must also setuserBot
within thecredentials
section of the settings.json totrue
.
Bot Permissions in Discord...
- In order to perform basic downloading functions, the bot will need
Read Message
permissions in the server(s) of your designated channel(s). - In order to respond to commands, the bot will need
Send Message
permissions in the server(s) of your designated channel(s). If executing commands via an Admin Channel, the bot will only needSend Message
permissions for that channel, and that permission will not be required for the source channel. - In order to process history commands, the bot will need
Read Message History
permissions in the server(s) of your designated channel(s).
How to Find Discord IDs...
- Discord Developer Mode: Enable
Developer Mode
in Discord settings underAppearance
. - Finding Channel ID: Enable Discord Developer Mode (see above), right click on the channel and
Copy ID
. - Finding User ID: Enable Discord Developer Mode (see above), right click on the user and
Copy ID
. - Finding Emoji ID: Enable Discord Developer Mode (see above), right click on the emoji and
Copy ID
. - Finding DM/PM ID: Inspect Element on the DM icon for the desired user. Look for
href="/channels/@me/CHANNEL_ID_HERE"
. Using this ID in place of a normal channel ID should work perfectly fine.
Features
- Supported File Downloading:
- Discord File Attachments
- Direct Links to Files
- Twitter (requires API key, see config section)
- Imgur (Single Posts & Albums)
- Flickr (requires API key, see config section)
- Google Drive (requires API Credentials, see config section)
- Tistory
- Streamable
- Gfycat
- Commands:
- Help (help - Alias: commands)
- Ping (ping - Alias: test)
- Status: Get an output of the current status of the bot (status - Alias: info)
- Stats: Have the bot dump stats (stats)
- [Must be Bot or Server Admin] History: Process all old messages in channel (history - Aliases: catalog, cache)
- [Must be Bot Admin] Exit (nice for process managers like pm2 for instant reload) (exit - Aliases: reload, kill)
Differences from Seklfreak's discord-image-downloader-go & Why I made this
Go 1.15 rather than 1.13I updated Sekl's project to match this.discordgo 0.22.0 rather than 0.16.1I updated Sekl's project to match this.- Implements dgrouter for commands
- Configuration is JSON-based rather than ini to allow more elaborate settings and better organization. With this came many features such as channel-specific settings.
- Channel-specific control of downloaded filetypes / content types (considers things like .mov as videos as well, rather than ignore them), Optional dividing of content types into separate folders.
- Download Support for Reddit
- (Optional) Reactions upon download success.
- (Optional) Discord messages upon encountered errors.
- Extensive bot status/presence customization.
- Consistent Log Formatting, Color-Coded Logging
- Somewhat different organization than original project; initially created from scratch then components ported over.
Fixed Compatability Issue withI updated Sekl's project to match this.xurls
that required people to edit the project, regardingxurls.Strict.FindAllString
. The issue was due to some people having xurls v2 installed while the projects go.mod required v1.1; changing go.mod to require v2 specifically seems to be the correct fix.
I've been a user of Seklfreak's project since ~2018 and it's been great for my uses, but there were certain aspects I wanted to expand upon, one of those being customization of channel configuration, and other features like message reactions upon success, differently formatted statuses, etc. If some aspects are rudimentary or messy, please make a pull request, as this is my first project using Go and I've learned everything from observation & Stack Overflow.
Guide: Saving History (Cataloging Old Messages)
This guide is to show you how to make the bot go through all old messages in a channel and catalog them as though they were being sent right now, in order to download them all.
You will need the Channel ID (see bottom of Setup) if attempting to catalog history from a specific channel or group of channels, within an admin channel.
ddg history
to catalog the current channel the command is sent in (must be registered inchannels
in settings).ddg history cancel
to stop cataloging the current channel the command is sent in (must be registered inchannels
in settings).ddg history <Channel ID(s)>
to catalog specified channels from within a designated Admin Channel (must be registered inadminChannels
in settings). You can do multiple channels per command if desired, separated by commas.ddg history cancel <Channel ID(s)>
to stop cataloging specified channels from within a designated Admin Channel (must be registered inadminChannels
in settings). You can do multiple channels per command if desired, separated by commas.
Guide: Settings / Configuration
I tried to make the configuration as user friendly as possible, though you still need to follow proper JSON syntax (watch those commas). All settings specified below labeled
[DEFAULTS]
will use default values if missing from the settings file, and those labeled[OPTIONAL]
will not be used if missing from the settings file.
When initially launching the bot it will create a default settings file if you do not create your own settings.json
manually. All JSON settings follow camelCase format.
If you have a config.ini
from Seklfreak's discord-image-downloader-go, it will import settings if it's in the same folder as the program.
Settings Examples
The following example is for a Bot Application (using a token), bound to 1 channel.
This setup exempts many options so they will use default values (see below). It shows the bare minimum required settings for the bot to function.
Example - Barebones settings.json:
{
"credentials": {
"token": "YOUR_TOKEN"
},
"channels": [
{
"channel": "DISCORD_CHANNEL_ID_TO_DOWNLOAD_FROM",
"destination": "FOLDER_LOCATION_TO_DOWNLOAD_TO"
}
]
}
Example - Selfbot settings.json:
{
"credentials": {
"email": "REPLACE_WITH_YOUR_EMAIL",
"password": "REPLACE_WITH_YOUR_PASSWORD"
},
"scanOwnMessages": true,
"presenceEnabled": false,
"channels": [
{
"channel": "DISCORD_CHANNEL_ID_TO_DOWNLOAD_FROM",
"destination": "FOLDER_LOCATION_TO_DOWNLOAD_TO",
"allowCommands": false,
"errorMessages": false,
"reactWhenDownloaded": false
}
]
}
Example - Advanced settings.json:
{
"credentials": {
"token": "YOUR_TOKEN",
"twitterAccessToken": "",
"twitterAccessTokenSecret": "",
"twitterConsumerKey": "",
"twitterConsumerSecret": ""
},
"admins": [ "YOUR_DISCORD_USER_ID", "YOUR_FRIENDS_DISCORD_USER_ID" ],
"adminChannels": [
{
"channel": "CHANNEL_ID_FOR_ADMIN_CONTROL"
}
],
"debugOutput": true,
"commandPrefix": "downloader_",
"allowSkipping": true,
"filterDuplicates": true,
"filterDuplicatesThreshold": 75,
"downloadRetryMax": 5,
"downloadTimeout": 120,
"githubUpdateChecking": true,
"presenceStatus": "dnd",
"presenceType": 3,
"presenceOverwrite": "{{count}} files",
"filenameDateFormat": "2006.01.02-15.04.05_",
"embedColor": "#29BEB0",
"inflateCount": 123456,
"channels": [
{
"channel": "THIS_CHANNEL_DOWNLOADS_EVERYTHING",
"destination": "EVERYTHING",
"overwriteEmbedColor": "#FF0000",
"userBlacklist": [ "USER_ID_FOR_PERSON_I_DONT_LIKE" ],
"divideFoldersByType": false,
"saveImages": true,
"saveVideos": true,
"saveAudioFiles": true,
"saveTextFiles": true,
"saveOtherFiles": true,
"savePossibleDuplicates": true,
"extensionBlacklist": [
".htm",
".html",
".php"
]
},
{
"channel": "THIS_CHANNEL_ONLY_DOWNLOADS_MEDIA",
"destination": "media",
"overwriteAllowSkipping": false,
"saveImages": true,
"saveVideos": true,
"saveAudioFiles": true,
"saveTextFiles": false,
"saveOtherFiles": false
},
{
"channel": "THIS_CHANNEL_ONLY_DOWNLOADS_MEDIA_FROM_ME_EXCEPT_DISCORD_FILES",
"destination": "media2",
"usersAllWhitelisted": false,
"userWhitelist": [ "MY_USER_ID" ],
"saveImages": true,
"saveVideos": true,
"saveAudioFiles": true,
"saveTextFiles": false,
"saveOtherFiles": false,
"domainBlacklist": [ "cdn.discordapp.com" ]
},
{
"channel": "THIS_CHANNEL_IS_STEALTHY",
"destination": "stealthy files",
"allowCommands": false,
"errorMessages": false,
"updatePresence": false,
"reactWhenDownloaded": false
},
{
"channels": [ "CHANNEL_1", "CHANNEL_2", "CHANNEL_3", "CHANNEL_4", "CHANNEL_5" ],
"destination": "lots of stuff",
"allowCommands": false,
"errorMessages": false,
"updatePresence": false
}
]
}
List of Settings
- credentials
[key/value object]
- token
[string]
- Required for Bot Login or User Login with 2FA, don't include if using User Login without 2FA.
- email
[string]
- Required for User Login without 2FA, don't include if using Bot Login.
- password
[string]
- Required for User Login without 2FA, don't include if using Bot Login.
[DEFAULTS]
userBot[bool]
- Default:
false
- Set to
true
for a User Login with 2FA, keep asfalse
if using a normal Bot.
- Default:
[OPTIONAL]
twitterAccessToken[string]
- Won't use Twitter API for fetching media from tweets if credentials are missing.
[OPTIONAL]
twitterAccessTokenSecret[string]
- Won't use Twitter API for fetching media from tweets if credentials are missing.
[OPTIONAL]
twitterConsumerKey[string]
- Won't use Twitter API for fetching media from tweets if credentials are missing.
[OPTIONAL]
twitterConsumerSecret[string]
- Won't use Twitter API for fetching media from tweets if credentials are missing.
[OPTIONAL]
flickrApiKey[string]
- Won't use Flickr API for fetching media from posts/albums if credentials are missing.
[OPTIONAL]
googleDriveCredentialsJSON[string]
- Path for Google Drive API credentials JSON file.
- Won't use Google Drive API for fetching files if credentials are missing.
- token
[OPTIONAL]
admins[array of strings]
- Array of User ID strings for users allowed to use admin commands
[OPTIONAL]
adminChannels[array of key/value objects]
- channel
[string]
- channel
[DEFAULTS]
debugOutput[bool]
- Default:
false
- Output debugging information.
- Default:
[DEFAULTS]
commandPrefix[string]
- Default:
"ddg "
- Default:
[DEFAULTS]
allowSkipping[bool]
- Default:
true
- Allow scanning for keywords to skip content downloading.
"skip", "ignore", "don't save", "no save"
- Default:
[DEFAULTS]
scanOwnMessages[bool]
- Default:
false
- Scans the bots own messages for content to download, only useful if using as a selfbot.
- Default:
[DEFAULTS]
filterDuplicateImages[bool]
- Default:
false
- Experimental feature to filter out images that are too similar to other cached images.
- Caching of image data is stored via a database file; it will not read all pre-existing images.
- Default:
[DEFAULTS]
filterDuplicateImagesThreshold[float64]
- Default:
0
- Threshold for what the bot considers too similar of an image comparison score. Lower = more similar (lowest is around -109.7), Higher = less similar (does not really have a maximum, would require your own testing).
- Default:
[OPTIONAL]
autorunHistory[bool]
- Autorun history for all registered channels in background upon launch.
- This can take anywhere between 2 minutes and 2 hours. It depends on how many channels your bot monitors and how many messages it has to go through. It can help to disable it by-channel for channels that don't require it (see
overwriteAutorunHistory
in channel options).
[DEFAULTS]
downloadRetryMax[int]
- Default:
3
- Default:
[DEFAULTS]
downloadTimeout[int]
- Default:
60
- Default:
[DEFAULTS]
githubUpdateChecking[bool]
- Default:
true
- Check for updates from this repo.
- Default:
[DEFAULTS]
presenceEnabled[bool]
- Default:
true
- Default:
[DEFAULTS]
presenceStatus[string]
- Default:
"idle"
- Presence status type.
"online"
,"idle"
,"dnd"
,"invisible"
,"offline"
- Default:
[DEFAULTS]
presenceType[int]
- Default:
0
- Presence label type. ("Playing <activity>", "Listening to <activity>", etc)
Game = 0, Streaming = 1, Listening = 2, Watching = 3, Custom = 4
- If Bot User, Streaming & Custom won't work properly.
- Default:
[OPTIONAL]
presenceOverwrite[string]
- Unused by Default
- Replace counter status with custom string.
- see Presence Placeholders for customization...
[OPTIONAL]
presenceOverwriteDetails[string]
- Unused by Default
- Replace counter status details with custom string (only works for User, not Bot).
- see Presence Placeholders for customization...
[OPTIONAL]
presenceOverwriteState[string]
- Unused by Default
- Replace counter status state with custom string (only works for User, not Bot).
- see Presence Placeholders for customization...
[DEFAULTS]
filenameDateFormat[string]
- Default:
"2006-01-02_15-04-05 "
- see this Stack Overflow post regarding Golang date formatting.
- Default:
[OPTIONAL]
embedColor[string]
- Unused by Default
- Supports
random
/rand
,role
/user
, or RGB in hex or int format (ex: #FF0000 or 16711680).
[OPTIONAL]
inflateCount[int]
- Unused by Default
- Inflates the count of total files downloaded by the bot. I only added this for my own personal use to represent an accurate total amount of files downloaded by previous bots I used.
[DEFAULTS]
numberFormatEuropean[bool]
- Default: false
- Formats numbers as
123.456,78
/123.46k
rather than123,456.78
/123,46k
.
- allChannels
[key/value objects]
- **Follow
channels
below for variables, except channel ID(s) are not necessary. - If a pre-existing config for the channel is not found, it will download from any and every channel it has access to, following these settings.
- **Follow
- allChannelsBlacklist
[array of strings]
- Unused by Default
- Blacklists channels (by ID) from
allChannels
.
- channels
[array of key/value objects]
[THIS OR BELOW]
channel[string]
- Channel ID to monitor.
[THIS OR ABOVE]
channels[array of strings]
- Channel IDs to monitor, for if you want the same configuration for multiple channels.
- destination
[string]
- Folder path for saving files, can be full path or local subfolder.
[DEFAULTS]
enabled[bool]
- Default:
true
- Toggles bot functionality for channel.
- Default:
[DEFAULTS]
allowCommands[bool]
- Default:
true
- Allow use of commands like ping, help, etc.
- Default:
[DEFAULTS]
errorMessages[bool]
- Default:
true
- Send response messages when downloads fail or other download-related errors are encountered.
- Default:
[DEFAULTS]
scanEdits[bool]
- Default:
true
- Check edits for un-downloaded media.
- Default:
[OPTIONAL]
overwriteAutorunHistory[bool]
- Overwrite global setting for autorunning history for all registered channels in background upon launch.
[DEFAULTS]
updatePresence[bool]
- Default:
true
- Update Discord Presence when download succeeds within this channel.
- Default:
[DEFAULTS]
reactWhenDownloaded[bool]
- Default:
true
- Confirmation reaction that file(s) successfully downloaded.
- Default:
[OPTIONAL]
reactWhenDownloadedEmoji[string]
- Unused by Default
- Uses specified emoji rather than random server emojis. Simply pasting a standard emoji will work, for custom Discord emojis use "name:ID" format.
[DEFAULTS]
blacklistReactEmojis[array of strings]
- Unused by Default
- Block specific emojis from being used for reacts. Simply pasting a standard emoji will work, for custom Discord emojis use "name:ID" format.
[DEFAULTS]
typeWhileProcessing[bool]
- Default:
true
- Shows " is typing..." while processing things that aren't processed instantly, like history cataloging.
- Default:
[OPTIONAL]
overwriteFilenameDateFormat[string]
- Unused by Default
- Overwrites the global setting
filenameDateFormat
(see above) - see this Stack Overflow post regarding Golang date formatting.
[OPTIONAL]
overwriteAllowSkipping[bool]
- Unused by Default
- Allow scanning for keywords to skip content downloading.
"skip", "ignore", "don't save", "no save"
[OPTIONAL]
overwriteEmbedColor[string]
- Unused by Default
- Supports
random
/rand
,role
/user
, or RGB in hex or int format (ex: #FF0000 or 16711680).
[DEFAULTS]
usersAllWhitelisted[bool]
- Default:
true
- Allow messages from all users to be handled. Set to
false
if you wish to useuserWhitelist
to only permit specific users messages to be handled.
- Default:
[OPTIONAL]
userWhitelist[array of strings]
- Use with
usersAllWhitelisted
asfalse
to only permit specific users to have their messages handled by the bot. Only accepts User ID's in the array.
- Use with
[OPTIONAL]
userBlacklist[array of strings]
- Use with
usersAllWhitelisted
as the defaulttrue
to block certain users messages from being handled by the bot. Only accepts User ID's in the array.
- Use with
[DEFAULTS]
divideFoldersByServer[bool]
- Default:
false
- Separate files into subfolders by server of origin (e.g. "My Server", "My Friends Server")
- Default:
[DEFAULTS]
divideFoldersByChannel[bool]
- Default:
false
- Separate files into subfolders by channel of origin (e.g. "my-channel", "my-other-channel")
- Default:
[DEFAULTS]
divideFoldersByUser[bool]
- Default:
false
- Separate files into subfolders by user who sent (e.g. "Me#1234", "My Friend#0000")
- Default:
[DEFAULTS]
divideFoldersByType[bool]
- Default:
true
- Separate files into subfolders by type (e.g. "images", "video", "audio", "text", "other")
- Default:
[DEFAULTS]
saveImages[bool]
- Default:
true
- Default:
[DEFAULTS]
saveVideos[bool]
- Default:
true
- Default:
[DEFAULTS]
saveAudioFiles[bool]
- Default:
false
- Default:
[DEFAULTS]
saveTextFiles[bool]
- Default:
false
- Default:
[DEFAULTS]
saveOtherFiles[bool]
- Default:
false
- Default:
[DEFAULTS]
savePossibleDuplicates[bool]
- Default:
false
- Default:
[DEFAULTS]
extensionBlacklist[array of strings]
- Default:
[ ".htm", ".html", ".php", ".exe", ".dll", ".bin", ".cmd", ".sh", ".py", ".jar" ]
- Ignores files containing specified extensions. Ensure you use proper formatting.
- Default:
[OPTIONAL]
domainBlacklist[array of strings]
- Ignores files from specified domains. Ensure you use proper formatting.
[OPTIONAL]
saveAllLinksToFile[string]
- Saves all sent links to file, does not account for any filetypes or duplicates, it just simply appends every raw link sent in the channel to the specified file.
Presence Placeholders
For presenceOverwrite
, presenceOverwriteDetails
, presenceOverwriteState
Key | Description |
---|---|
{{dgVersion}} |
discord-go version |
{{ddgVersion}} |
Project version |
{{apiVersion}} |
Discord API version |
{{countNoCommas}} |
Raw total count of downloads (without comma formatting) |
{{count}} |
Raw total count of downloads |
{{countShort}} |
Shortened total count of downloads |
{{numGuilds}} |
Number of guilds bot is a member of |
{{numChannels}} |
Number of bound channels |
{{numAdminChannels}} |
Number of admin channels |
{{numAdmins}} |
Number of designated admins |
{{timeSavedShort}} |
Last save time formatted as 3:04pm |
{{timeSavedShortTZ}} |
Last save time formatted as 3:04pm MST |
{{timeSavedMid}} |
Last save time formatted as 3:04pm MST 1/2/2006 |
{{timeSavedLong}} |
Last save time formatted as 3:04:05pm MST - January 2, 2006 |
{{timeSavedShort24}} |
Last save time formatted as 15:04 |
{{timeSavedShortTZ24}} |
Last save time formatted as 15:04 MST |
{{timeSavedMid24}} |
Last save time formatted as 15:04 MST 2/1/2006 |
{{timeSavedLong24}} |
Last save time formatted as 15:04:05 MST - 2 January, 2006 |
{{timeNowShort}} |
Current time formatted as 3:04pm |
{{timeNowShortTZ}} |
Current time formatted as 3:04pm MST |
{{timeNowMid}} |
Current time formatted as 3:04pm MST 1/2/2006 |
{{timeNowLong}} |
Current time formatted as 3:04:05pm MST - January 2, 2006 |
{{timeNowShort24}} |
Current time formatted as 15:04 |
{{timeNowShortTZ24}} |
Current time formatted as 15:04 MST |
{{timeNowMid24}} |
Current time formatted as 15:04 MST 2/1/2006 |
{{timeNowLong24}} |
Current time formatted as 15:04:05 MST - 2 January, 2006 |
{{uptime}} |
Shortened duration of bot uptime |
Info for Developers
- I'm a complete amateur with Golang. If anything's bad please make a pull request.
- Versioning is
[MAJOR].[MINOR].[PATCH]
- I try to be consistent with annotation but it's not perfect.
- Logging generally follows certain standards and patterns with formatting and color-coding.
Documentation ¶
There is no documentation for this package.