robotmonitor 0.1.0

Description:

robotmonitor 0.1.0

robotmonitor
Maintain datalake to monitor robot real time situation
Project Setup Instructions
Project Overview
This project aims to connect robot logs to the internet, ensuring comprehensive
log management and instant error notification. The objectives include:

Saving all logs in a NoSQL data store.
Generating instant messages upon detecting errors.

We achieve this through four main steps:

Configure Slack for notifications.
Configure folder sharing from the robot to a Linux system.
Install and configure the Elastic Stack (ELK).
Run the Python service to handle logs and notifications.

Step-by-Step Instructions
##########################
1. Configure Slack
##########################
Slack Configuration for Notifications
This section guides you through setting up Slack to receive instant messages
when errors are detected in robot logs. Follow the steps below to create and
configure a Slack app, which will allow you to send notifications directly
to your chosen Slack channel.
Step 1.1: Create and Configure a Slack App


Create a Slack App:

Navigate to Slack API and
click on "Create New App." (you need to sign in first).
Select "From scratch," name your app, and choose the Slack workspace
where notifications will be sent.
Click "Create App."



Set Permissions:

Go to "OAuth & Permissions" (left side, under Features)
Scroll down to "Scopes" and click "Add an OAuth Scope."
Add the following scopes to enable message sending and channel reading:

chat:write
channels:read
groups:read
im:read
mpim:read
channels:history
conversations.connect:read
incoming-webhook


Scroll back up to "Install to Workspace" (green) to generate and obtain the
OAuth Access Token, which will be used to authenticate API requests.
Create a configuration file named .env with the following line:
SECONDS_TO_STAY_SILENCE=3
SLACK_TOKEN=xoxb-******



Incoming Webhook Setup (Optional)
You can skip this step. It allows another method to post to slack, called
webhook. Webhook is simpler, but with a limited functionality.

In the app settings, select "Incoming Webhooks" from the sidebar.
Toggle the activation switch to "On."
Click "Add New Webhook to Workspace" and select the channel where you want
to receive notifications.
Copy the generated Webhook URL, which will be used in your Python script
to send messages.



Step 1.2: Find Your Slack Channel ID

Locate Channel ID:


Open your Slack workspace in a browser.
Navigate to the channel where notifications should appear.
The URL will include the Channel ID at the end, e.g.,
https://app.slack.com/client/T00000000/C00000000 — C00000000
is your Channel ID.
Add to .env the following line:
SLACK_CHANNEL_ID=C***



Invite the bot to the channel.
Go to your Slack Channel and type the following as a message.
/invite @bot-name


Test
pip install -r requirements.txt
python tests/test_slack_bot.py


##########################
2. Folder Sharing
##########################
On the Computer connected to the robot:


Navigate to the Logs Folder:

Go to c:\Program Files (x86)\Hamilton\



Configure Sharing:

Right-click the Logs folder and select Properties.
Switch to the Sharing tab and click on Advanced Sharing.
Check Share this folder. Set the share name, for instance, RobotLogs.
Click on Permissions and ensure that you grant 'Read' or 'Full Control'
permissions, depending on the required level of access.
Confirm all settings by clicking OK.



Note the Network Path:

The shared folder's network path will typically look
like \\YOUR-PC-NAME\RobotLogs.
Ensure to note this path as it will be used for mounting the folder on
Ubuntu.



On Ubuntu:


install cifs
sudo apt-get install cifs-utils


Create the folder for sharing
sudo mkdir /var/log/shared_logs


Mount temporarily:
Use the following command to mount the Windows shared folder to the mount point you created:
sudo mount -t cifs -o username=YOUR_WINDOWS_USERNAME,password=YOUR_WINDOWS_PASSWORD,iocharset=utf8,noperm //YOUR-PC-NAME/RobotLogs /var/log/shared_logs


Configure Persistent Mounting
Add the following line to /etc/fstab (you can use emacs/nano/vi)
//YOUR-PC-NAME/RobotLogs /var/log/shared_logs cifs username=YOUR_WINDOWS_USERNAME,password=YOUR_WINDOWS_PASSWORD,iocharset=utf8,noperm 0 0


remount:
sudo mount -a


#######################
3. Install ELK
#######################
Installtion of the ElasticSearch,Kibana,Logstash is out of the scope of this repo.
You can use paid services such as https://logz.io/, https://www.elastic.co/,
https://aws.amazon.com/opensearch-service/ or dare to install it yourself !
See e.g. here for the last option:
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-22-04
I have provided in this repo a logstash configution file
scripts/logstash.conf
To load all log files into Elasticsearch.
Make sure to update the user and password in the relevant section.
##############################
4. run the notifier script
##############################
nohup python3 scripts/elasticsearch_error_notifier.py >> elasticsearch_error_notifier.log 2>&1 &
References

For a comprehensive guide on Slack APIs and further configuration details,
visit the Slack API Documentation.
For specifics on message sending methods, see
Sending Messages on Slack.

Note
With the above setup, your project can send real-time notifications to a
Slack channel, enhancing the monitoring system's responsiveness to issues.
Depending on your preference and requirement, choose between using the API
for complex interactions or Webhooks for straightforward message sending.