simple-useragent 0.1.5

Description:

simpleuseragent 0.1.5

Fetches real world, up-to-date user agents for use in web scraping to avoid bot detection.
No more fake or outdated user agents, only user agents of real users.
You can either get random or usage-weighted user agents. It caches
the user agents locally to avoid unnecessary API calls, and refreshes them automatically every 24 hours
from the public API of useragents.me. If you need full-fledge fake browser
headers including client hints like sec-ch-ua, sec-fetch-site and more, check out my
simple-header package.






 

Contents

Features
Installation
Usage

Quickstart
Advanced Usage
Settings and Parameters


Development
Contributors
Credits
License

 

Features

Up-to-date: No fake or outdated user agents. Only real world data, refreshed every 24 hours.
Wide Support: User Agents for Windows, macOS, Linux, Android and iOS devices: Google Chrome, Firefox, Safari, Edge, Opera, Whale and QQ browsers.
Lightweight: Caches user agents locally to consume minimal system resources.
Simple: Easy to use and understand with a clean and simple API.
Compatible: Supports Python 3.8 and above. Runs on Windows, macOS and Linux.
Tested: Has 99% test coverage and is continuously tested.
Privacy: Protects the user by not collecting or sending any personal data.
Open Source: Provides transparency and allows community contributions for continuous development.

 

Installation
Just install the package from PyPi using pip:
pip install simple-useragent

 

Usage
Quickstart
Just import the package and use the convenience functions. For more advanced usage, you can initialize the class to set custom settings.
import simple_useragent as sua

sua.get(num=2, mobile=True) # List of the 2 most common mobile user agents (attributes explained below).
# [UserAgent('Mozilla/5.0 (Android ...'), UserAgent('Mozilla/5.0 (iPhone; ...')]

sua.get_list(shuffle=True, force_cached=True) # Random list of available desktop user agents strings.
# ['Mozilla/5.0 ...', 'Mozilla/5.0 (iPhone ...', 'Mozilla/5.0 (iPhone ...', ...]

sua.get_dict() # Dictionary with all desktop and mobile user agents.
# {'desktop': ['Mozilla/5.0 ...', ...] 'mobile': ['Mozilla/5.0 (iPhone ...', ...]}

 
Advanced Usage
Import the package and initialize the UserAgents class to set custom settings (optional, see Settings and Parameters for details).
import simple_useragent as sua

simple_ua = sua.UserAgents(max_retries=3, timeout=5, cache_duration=86400, cache_location='example/path/to/folder')

 
Fetching User Agents.
# Fetch a specified number of random mobile user agent instances (with settings from the class above).
simple_ua.get(num=2, shuffle=True, mobile=True)
# [UserAgent('Mozilla/5.0 (iPhone ...'), UserAgent('Mozilla/5.0 (iPhone; ...')]

 
You can also use the convenience functions to get user agents without initializing the class.
sua.get(num=2, mobile=True) # List of the 2 most common mobile user agent (attributes explained below).
# [UserAgent('Mozilla/5.0 (Android ...'), UserAgent('Mozilla/5.0 (iPhone; ...')]

sua.get_list(force_cached=True) # List of all available desktop user agents as strings.
# ['Mozilla/5.0 ...', 'Mozilla/5.0 (iPhone ...', 'Mozilla/5.0 (iPhone ...', ...]

sua.get_dict() # Dictionary with all desktop and mobile user agents.
# {'desktop': ['Mozilla/5.0 ...', ...] 'mobile': ['Mozilla/5.0 (iPhone ...', ...]}

 
The instance offers attributes for the user agent properties.
# Parse a custom string directly to the UserAgent class and access its attributes.
obj = sua.parse('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36')
obj.string # 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit ...'
obj.browser # 'Chrome', 'Firefox', 'Safari', 'Edge', 'IE', 'Opera', 'Whale', 'QQ Browser', 'Samsung Browser', 'Other'
obj.browser_version # '110', '109', '537', ...
obj.browser_version_minor # '0', '1', '36', ...

# You can also access the attributes with square brackets.
obj = sua.get(num=1, shuffle=True)[0]
obj['os'] # 'Windows', 'macOS', 'Linux', 'Android', 'iOS', 'Other'
obj['os_version'] # '10', '7', '11', '14', ...
obj['os_version_minor'] # '0', '1', '2', ...
obj['mobile'] # True / False

 
Settings and Parameters
The functions can take the following parameters:

num: The number of user agents to fetch (default: None = gets you all user agents available).
mobile: Fetch mobile or desktop user agents (default: False = desktop).
shuffle: Whether to shuffle/randomize the order of user agents (default: False = ordered by usage).
force_cached: Force the use of memory or file cached user agents (default: None = fetches new user agents if cache is outdated, False = always call the API, True = always use the cache).

 
You can set custom preferences when initializing the class with UserAgents(...).

max_retries: The maximum number of retries to reach the API, before falling back to local cache (default: 3).
timeout: The timeout in seconds for the API request (default: 5).
cache_duration: The duration in seconds for the user agents to be cached (default: 86400 = 1 day).
cache_location: The folder in which the user agents are cached, specific to the OS. You can see the default location with UserAgents._cache_location.

 

Notes:

The user agents are cached locally to avoid unnecessary API calls, and are refreshed automatically every 24 hours.
During runtime the user agents are stored in memory and written to a cache file for persistence and performance.
Every time you invoke a simple-useragent function, it is automatically checked for outdated user agents.


 

Development
As an open-source project, I strive for transparency and collaboration in my development process. I greatly
appreciate any contributions members of our community can provide. Whether you are fixing bugs, proposing features,
improving documentation, or spreading awareness - your involvement strengthens the project. Please review the
code of conduct to understand how we work together
respectfully.

Bug Report: If you are experiencing an issue while using the package, please create an issue.
Feature Request: Make this project better by submitting a feature request.
Documentation: Improve our documentation by adding a wiki page.
Community Support: Help others on GitHub Discussions.
Security Report: Report critical security issues via our template.

 

Contributors
Thank you so much for giving feedback, implementing features and improving the code and project!



 

Credits
Full credits are in the ACKNOWLEDGMENTS file.
 

License
Provided under the terms of the GNU GPL3 License © Lennart Haack 2024.
See LICENSE file for details.
For the licenses of used third party libraries and software, please refer to the ACKNOWLEDGMENTS file.