pkgraph

Description:

pkgraph

pkgraph #
This tool allows the user to gather Dart package data from a pub server or
local package and load the dependency graph into a Neo4j database. It is
a work-in-progress, additional documentation will be forthcoming.
For now, it works, though it isn't terrible efficient.
Running #
To run, just give it a package name and let it churn for awhile.
Note that right now you'll need to have Neo4j running on localhost,
and it will need to have authentication turned off. There is a script
in the scripts/ directory that will run Neo4j, appropriately
configured, in a Docker container.
pub run pkgraph w_common
copied to clipboard
It is also possible to start your traversal at a local package or
application that may not be published to a pub server. Just pass
the --local flag and any package names you pass will be interpreted
as local paths to directories that contain pubspec.yaml files,
presumably Dart packages.
pub run pkgraph --local projects/secret_dart_app
copied to clipboard
If you're interested in the actual dependency versions resulting from the pub
solver, it is possible to create SOLVED_TO edges between the version of the
package you're interested in and the solved dependency versions. Note that it
is only possible to do this with a local package since it requires the
pubspec.lock file.
pub run pkgraph --local --solved projects/secret_dart_app
copied to clipboard
Example Queries #
Once you have some data loaded into a Neo4j database, what can you do
with it? There are some examples that might be useful or inspirational.
match (:Package {name: "w_common"})
-[:HAS_VERSION]->(:Version)-[:MAY_USE]->(:Version)<-[:HAS_VERSION]-
(d:Package)
return distinct d.name
copied to clipboard
The query above will find all packages that are depended upon by at
least one version of the w_common package.
match (:Source {url: "https://pub.dartlang.org"})<-[:HOSTED_ON]-
(:Package {name: "state_machine"})-[:HAS_VERSION]->
(state_machine:Version)-[:MAY_USE]->
(w_common:Version)<-[:HAS_VERSION]-
(:Package {name: "w_common"})
return state_machine.version as state_machine,
collect(w_common.version) as w_common
copied to clipboard
This query will display a table of each available version of the
state_machine that depends on the w_common package along with the
possible versions of the w_common package that it can use.
match (p:Package)-[:HAS_VERSION]-(v:Version)
where v.dart2 = true
return p.name as package, collect(v.version) as versions
order by p.name asc
copied to clipboard
Dart 2 is out, have you heard? This query returns all the packages
that support Dart 2 and a list of specific versions that for each.
Known Issues #
In theory, the queries the tool runs should be idempotent, and should just
update nodes with new data. This means you should be able to run it against
an already-populated database without any trouble.
However, I haven't verified this and there are no tests for it, so it is
probably safer to blow away your database before re-running. Or you could add
a test suite, whatever suits you.
Future Work #
In addition to the work items listed below, there are, let's say ample,
additional items sprinkled throughout the source code.

Handle git dependencies
Parse author strings when possible to separate the name and email
Allow a combination of local, pub, and even git packages
Serialize the package version cache for reuse on subsequent runs

Feel free to report issues if you find bugs or have suggestions to
improve the tool.
Roadmap #
Eventually the tool should be able to run as a cron job and pull in a variety
of packages of mixed type (local, public, and private). I'd like it to also
be able to execute a handful of potentially useful queries and display
results in a reasonable fashion. A nifty web interface to view those results
and maybe even control the tool itself would be bonus points.