wagtail-content-audit 0.1

Description:

wagtailcontentaudit 0.1

wagtail-content-audit
Content audit utilities for Wagtail. Still a work in progress.
For Wagtail sites with deeply-nested blocks and a large amount of potentially old content, it can be helpful to inspect block usage and be able to search through the content as it exists in the database. This library is intended to help with these and other challenges of auditing the content in Wagtail.

Dependencies
Installation
Usage

Block usage

Block usage management command
Block usage QuerySet


Page search

Page search management command
Page search QuerySet




Getting help
Getting involved
Licensing

Dependencies

Python 3.8+
Django 3.2 (LTS)+
Wagtail 5.2 (LTS)+

It should be compatible at all intermediate versions, as well.
If you find that it is not, please file an issue.
Installation

Install wagtail-content-audit:

pip install wagtail-content-audit


Add wagtail_content_audit as an installed app in your Django settings.py:

INSTALLED_APPS = (
...
"wagtail_content_audit",
...
)

Usage
wagtail-content-audit provides two primary audit tools at present:

Block usage auditing
Page field searching

For both, it provides a QuerySet-like object using queryish that returns instances of a dataclass with relevant result data.
Block usage
Block usage is intended to audit deeply-nested Wagtail Blocks to discover how much these blocks might be used, and wwithin which other blocks and fields that usage occurs.
Block usage management command
wagtail-content-audit provides a management command to run the block usage audit and output CSV results:
./manage.py block_usage

The resulting CSV can be redirected to a file:
./manage.py block_usage > block_usage_audit.csv

The command takes the following arguments:
--pagetype PAGETYPE_AND_FIELD, -p PAGETYPE_AND_FIELD
Limits the audit to the particular page type(s) and Wagtail StreamField as a dotted path. For example,
./manage.py block_usage --pagetype myapp.PageWithContent.content

Will output the blocks used in all myapp.PageWithContent pages' content field.
Block usage QuerySet
from wagtail_content_audit.query import BlockUsageQuerySet

The underlying queryish QuerySet can be used outside of the management management command as well. This QuerySet behaves like any queryish QuerySet, with a limited set of available options.
It can be filtered for page types:
filtered_queryset = BlockUsageQuerySet().filter(page_model="myapp.PageWithContent")

It can be filtered for Wagtail StreamFields:
filtered_queryset = BlockUsageQuerySet().filter(field="content")

And these can be combined:
filtered_queryset = BlockUsageQuerySet().filter(page_model="myapp.PageWithContent", field="content")

The queryset can also be sliced:
sliced_queryset = BlockUsageQuerySet()[:5]

The resulting objects in the queryset are wagtail_content_audit.query.AuditedBlock objects with the following schema:
@dataclass
class AuditedBlock:
page_model: type
field: str
path: str
block: type
pages: list
total_occurrences: int = 0
pages_count: int = 0
pages_live_count: int = 0
pages_in_default_site_count: int = 0

Page search
Page search is intended to enable searching for specific patterns (using regular expressions) in text content in all Wagtail Page model fields.
For StreamFields specifically, it returns explicit block paths within a StreamField (i.e., 0.list.item.1.richtext for a result found in the second child list item in the first child block in the field) as well as the general block path (i.e., list.item.richtext) so that the blocks can be targetted using Wagtail StreamField migrations.
Page search management command
wagtail-content-audit provides a management command to run the page search audit and output CSV results:
./manage.py page_search -s '[tT]est'

The resulting CSV can be redirected to a file:
./manage.py page_search -s '[tT]est' > page_search_test.csv

The command takes the following arguments:
--pagetype PAGETYPE_AND_FIELD, -p PAGETYPE_AND_FIELD
Limits the search to the particular page type(s) and model field as a dotted path. For example,
./manage.py page_search -s '[tT]est' --pagetype myapp.PageWithContent.content

Will only search within the content field of myapp.PageWithContent pages.
Page search QuerySet
from wagtail_content_audit.query import PageSearchQuerySet

The underlying queryish QuerySet can be used outside of the management management command as well. This QuerySet behaves like any queryish QuerySet, with a limited set of available options.
It can be searched with any regular expression string:
search_queryset = PageSearchQuerySet().filter(search=r"[tT]est")

It can be filtered for page types:
filtered_queryset = PageSearchQuerySet().filter(search=r"[tT]est", page_model="myapp.PageWithContent")

It can be filtered for model fields:
filtered_queryset = PageSearchQuerySet().filter(search=r"[tT]est", field="content")

And these can be combined:
filtered_queryset = PageSearchQuerySet().filter(search=r"[tT]est", page_model="myapp.PageWithContent", field="content")

The queryset can also be sliced:
sliced_queryset = BlockUsageQuerySet()[:5]

The resulting objects in the queryset are wagtail_content_audit.query.pagesearch.PageMatch objects with the following schema:
@dataclass
class PageMatch:
page_model: type
page: Page
field_name: str
field_type: str
stream_field_path: list
block_type: type
result_path: list
matches: list

Getting help
Please add issues to the issue tracker.
Getting involved
General instructions on how to contribute can be found in CONTRIBUTING.
Licensing

TERMS
LICENSE
CFPB Source Code Policy