ao3statscraper 0.1.8

Description:

ao3statscraper 0.1.8

AO3StatScraper
AO3StatScraper is a small python package that provides command line scripts to
fetch your AO3 (Archive Of Our Own) statistics to store and display them.
Installation
AO3StatScraper is available on
PyPI, and can obtained any regular
way you'd install a python package, e.g. using pip, conda, etc.
For example, using pip:
On Unix/OSX:
python3 -m pip install ao3statscraper

On Windows:
py -m pip install ao3statscraper

Alternately, the source code is available on
gitlab.
you can install a local version by cloning the repository using git.
On OSX and Linux, you can use:
git clone https://gitlab.com/athenaslilowl1/AO3StatScraper.git # get you the git repository
cd AO3StatScraper # go into the directory
python3 -m pip install . # install package

User Guide
Overview: How It Works
AO3StatScraper simply downloads your AO3 stats page (pretty much the same as
opening it in your browser and hitting Save this page...) and then extract the
stats from that html content. To be able to access your stats, you need to log
in to AO3, which is why AO3StatScraper will ask you for your username and
password.
By default AO3StatScraper (i.e. the script ao3get) will store your current
stats as a snapshot. This includes both your total user statistics as well as
your individual work statistics. The data is written in
the csv (Comma Separated Values) format, so plenty of other software and
packages should be able to read it in easily. Your total user statistics are
stored in a separate file from your work statistics.
Getting Started
The main use case for AO3StatScraper is to fetch and display your current AO3
statistics using scripts provided by AO3StatScraper. Currently, there are 5
scripts:

ao3get : The main script to fetch and display your AO3 stats.
ao3plot: Plots the stats stored with ao3get.
ao3diff: Show the difference in stats compared to an earlier date.
ao3_hits_to_kudos: List all your works in ascending order of their
hits/kudos ratio
ao3_purge: Deletes saved stats such that there is a minumum time between the
remaining ones

The scripts are discussed in more detail further below.
IMPORTANT: Before you can run the other scripts, you first need to configure
AO3StatScraper. This is done by calling ao3get. It will launch the
configuration dialogue automatically if it hasn't been configured yet. You can
always re-configure it by invoking ao3get -c.
ao3get
This is the main script to fetch and store your AO3 stats. There are several
running modes. When in doubt, invoke ao3get --help to see the available
options.
The default running mode is --diff.

--all: Fetch and store current stats from AO3 into a snapshot, and display
stats for all works.
--repeat: Don't fetch new stats, but show the changes between the last two
stored stats snapshots.
--diff: Fetch and store current stats from AO3 into a snapshot, and display
only stats for works that have changed since the last snapshot. If there were
no changes in work stats, it will display only the user's total stats.
--config: Run the configuration dialogue and exit.
--remove-last: Deletes the last snapshot ao3get stored and exits. May come
in handy if you're nervously re-downloading your stats every minute.
--no-write: This flag modifies the behavious for --all and --diff
running modes. While current stats are fetched from AO3, they won't be written
into a snapshot.

The list above does not cover all the available options. Please use
ao3get --help to see all the available options.
Using ao3get requires you to log in to your AO3 account. You can either type
in your username and password each time you invoke it, or you can store it with
AO3StatScraper, locked behind a master password. There are no restrictions on
how sophisticated your master password needs to be, so if you can't be
inconvenienced, you can even leave it empty. Alternately, you can opt out of
using a master password at all, although this is not encouraged. But it would
allow you to e.g. set up an automated way to fetch your stats at regular times.
ao3plot
ao3plot will display some simple graphs based on the stored snapshots. It
never stores snapshots itself, you will need to do that using ao3get.
By default, ao3plot will ask you to select which work you would like to see
graphs of stats for. You can also select to plot your total user statistics.
Alternately, you can skip that dialogue by using the following flags:

-u: Show total user statistics.
-i [ID]: Show the statistics for the work with AO3 ID [ID]. For example,
if your work is under the link https://archiveofourown.org/works/24280306,
the ID would be 24280306.
<number>: Show the statistics for the work with index <number>, where the
index refers to the number of the work as shown in the selection dialogue
when running ao3plot without any command line arguments.

It is possible that the stats graphs aren't displayed nicely on all screens. If
that is the case for you, you may want to try the --no-prettify flag to obtain
a bare-bones plot without any prettifications. It may not look as nice, but at
least you should be able to see the data.
Finally, you can also try the --diff option to also display the difference
in stats over time. By default, it shows the weekly differences between the
stats.
ao3diff
Fetch (but do not store) your AO3 stats and compare them to stats from the past.
It never stores snapshots itself, you will need to do that using ao3get.
By default, it will compare to your stats from a week ago, but you can also
select a day, a week, a month, or a year back using the -d, -w, -m, or -y
flag, respectively. Alternately, you can specify a date in the YYYY-MM-DD
format.
ao3_hits_to_kudos
This script just reads in the last stored snapshot and prints out all your works
in ascending order of their hits/kudos ratio.
ao3_purge
In case you find yourself in a situation where you feel you have stored way too
many snapshots, ao3_purge offers you the option to delete stats snapshots such
that there is some minimal time between them. By default, this frequency is set
to 12h. You can provide the frequency you like using the --frequency flag.
Examples
This is example output what ao3get will show you when you run it:

It will only list the works that have changes since you last cheked (i.e. stored
a snapshot).
But you can also view all your works:

Plotting the total user statistics with ao3plot will show you something like
this:

Running ao3plot for a specific work will give you 2 plots:


Having issues?
Please let me know by raising an issue on
gitlab.
I'm using unix exclusively, so there may be issues on other operating systems.
Testers and devs on OSX and windows are also very welcome to let me know what's
working and what needs fixing.
Roadmap and Contributing
Help and contributions to maintain and extend this tool are very welcome!
Some ideas what might be added in the future include

Writing/reading of file formats other than csv.
Maybe add a GUI for CLI-averse users. This tool was always intended to be a
command line tool on my end. Having never programmed a GUI in my life, I don't
intend to start now. However, if anybody is willing to pack this up in a nice
simple portable GUI, you are very welcome and encouraged to do so! I'll gladly
add it to the repository.

I'm using unix exclusively, so there may be issues on other operating systems.
Testers and devs on OSX and windows are also very welcome to let me know what's
working and what needs fixing.
Acknowledgements
Parts of the AO3 fetching mechanism is lifted from the AO3 API
package.