simple-data-collector 0.2.0

Description:

simpledatacollector 0.2.0

Simple Data Collector










Thread-safe and atomic collection of tabular data into csv-files.


The simple-data-collector provides a single class with with following methods:

save
append
load
remove

It was created as a utility function for the Hyperactive-package. It was intended to be used as a search-data collection tool. The search-data can be collected during the optimization run as a dictionary via append or after the run as a dataframe with the save-method.
The append-method is thread-safe to work with hyperactive-multiprocessing. The save-method is atomic to avoid accidental data-loss.
The simple-data-collector handles functions in the data by converting them to strings. If the data is loaded you can pass the search_space to convert the strings back to functions.

Installation
pip install simple-data-collector


Example
import numpy as np
from hyperactive import Hyperactive
from simple_data_collector import DataCollector

collector = DataCollector("./search_data.csv") # the csv is created automatically


def ackley_function(para):
x, y = para["x"], para["y"]

loss = (
-20 * np.exp(-0.2 * np.sqrt(0.5 * (x * x + y * y)))
- np.exp(0.5 * (np.cos(2 * np.pi * x) + np.cos(2 * np.pi * y)))
+ np.exp(1)
+ 20
)

data_dict = para.para_dict
data_dict["score"] = -loss
collector.append(data_dict) # you can append a dictionary to the csv

return -loss


search_space = {
"x": list(np.arange(-10, 10, 0.01)),
"y": list(np.arange(-10, 10, 0.01)),
}


hyper = Hyperactive()
hyper.add_search(ackley_function, search_space, n_iter=3000)
hyper.run()
search_data = hyper.search_data(ackley_function)

# collector.save(search_data) # save a dataframe instead of appending a dictionary

search_data_l = collector.load(search_space) # load data

print(search_data_l)