Skip to content

Commit

Permalink
Allow us to pass configuration options in a json file
Browse files Browse the repository at this point in the history
Addresses #48
  • Loading branch information
nanos committed Jun 9, 2023
1 parent d4dfa1e commit 5a3db44
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 2 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
.vscode/launch.json
artifacts/*
config.json
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,20 @@ When using a cronjob, we are using file based locking to avoid multiple overlapp

If you are running FediFetcher locally, my recommendation is to run it manually once, before turning on the cron job: The first run will be significantly slower than subsequent runs, and that will help you prevent overlapping during that first run.

When running FediFetcher locally, it may be advantageous to supply a json file of configuration options, instead of supplying a long list of command line flags. To do so, create a json file with your configuration options, e.g.

```json
{
"server": "mstdn.thms.uk",
"access-token": "{token}",
"home-timeline-length": 200,
"max-followings": 80,
"from-notifications": 1
}
```

and then run your script like so: `python find_posts.py --config=path/to/json`.

*Note:* if you wish to run FediFetcher using Windows Task Scheduler, you can rename the script to the `.pyw` extension instead of `.py`, and it will run silently, without opening a console window.

### 2.3) Run FediFetcher from a container
Expand Down Expand Up @@ -110,6 +124,7 @@ Please find the list of all configuration options, including descriptions, below

| Environment Variable Name | Command line flag | Required? | Notes |
|:---------------------------------------------------|:----------------------------------------------------|-----------|:------|
| -- | `--config` | No | You can use this to point to a JSON file containing your configuration options, instead of supplying configuration options as command line flags.
| -- | `--access-token` | Yes | The access token. If using GitHub action, this needs to be provided as a Secret called `ACCESS_TOKEN`. If running as a cron job or a container, you can supply this argument multiple times, to [fetch posts for multiple users](https://blog.thms.uk/2023/04/muli-user-support-for-fedifetcher) on your instance. |
|`MASTODON_SERVER`|`--server`|Yes|The domain only of your mastodon server (without `https://` prefix) e.g. `mstdn.thms.uk`. |
| `HOME_TIMELINE_LENGTH` | `--home-timeline-length` | No | Provide to fetch remote replies to posts in the API-Key owner's home timeline. Determines how many posts we'll fetch replies for. Recommended value: `200`.
Expand Down
23 changes: 21 additions & 2 deletions find_posts.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python3

from datetime import datetime, timedelta
import string
from dateutil import parser
import itertools
import json
Expand All @@ -14,8 +15,9 @@

argparser=argparse.ArgumentParser()

argparser.add_argument('--server', required=True, help="Required: The name of your server (e.g. `mstdn.thms.uk`)")
argparser.add_argument('--access-token', action="append", required=True, help="Required: The access token can be generated at https://<server>/settings/applications, and must have read:search, read:statuses and admin:read:accounts scopes. You can supply this multiple times, if you want tun run it for multiple users.")
argparser.add_argument('-c','--config', required=False, type=str, help='Optionally provide a path to a JSON file containing configuration options. If not provided, options must be supplied using command line flags.')
argparser.add_argument('--server', required=False, help="Required: The name of your server (e.g. `mstdn.thms.uk`)")
argparser.add_argument('--access-token', action="append", required=False, help="Required: The access token can be generated at https://<server>/settings/applications, and must have read:search, read:statuses and admin:read:accounts scopes. You can supply this multiple times, if you want tun run it for multiple users.")
argparser.add_argument('--reply-interval-in-hours', required = False, type=int, default=0, help="Fetch remote replies to posts that have received replies from users on your own instance in this period")
argparser.add_argument('--home-timeline-length', required = False, type=int, default=0, help="Look for replies to posts in the API-Key owner's home timeline, up to this many posts")
argparser.add_argument('--user', required = False, default='', help="Use together with --max-followings or --max-followers to tell us which user's followings/followers we should backfill")
Expand Down Expand Up @@ -750,6 +752,23 @@ def toJSON(self):

arguments = argparser.parse_args()

if(arguments.config != None):
if os.path.exists(arguments.config):
with open(arguments.config, "r", encoding="utf-8") as f:
config = json.load(f)

for key in config:
setattr(arguments, key.lower().replace('-','_'), config[key])

else:
log(f"Config file {arguments.config} doesn't exist")
sys.exit(1)

if(arguments.server == None or arguments.access_token == None):
log("You must supply at least a server name and an access token")
sys.exit(1)


runId = uuid.uuid4()

if(arguments.on_start != None and arguments.on_start != ''):
Expand Down

0 comments on commit 5a3db44

Please sign in to comment.