gcloud

Description:

gcloud

The gcloud package provides a high level idiomatic Dart interface to some of
the most widely used Google Cloud Platform services.
Google Cloud Platform support package (gcloud) #
Currently the following services are supported:

Cloud Datastore
Cloud Storage
Cloud Pub/Sub

Status: Experimental #
NOTE: This package is currently experimental and published under the
labs.dart.dev pub publisher in order to
solicit feedback.
For packages in the labs.dart.dev publisher we generally plan to either graduate
the package into a supported publisher (dart.dev, tools.dart.dev) after a period
of feedback and iteration, or discontinue the package. These packages have a
much higher expected rate of API and breaking changes.
Your feedback is valuable and will help us evolve this package. For general
feedback, suggestions, and comments, please file an issue in the
bug tracker.
API details #
The APIs in this package are all based on the generic generated APIs in the
googleapis and googleapis_beta packages.
This means that the authentication model for using the APIs in this package
uses the googleapis_auth package.
Note that this package is only intended for being used with the standalone VM
in a server or command line application. Don't expect this package to work on
the browser or in Flutter.
The code snippets below demonstrating the use of this package all assume that
the following imports are present:
import 'dart:io';

import 'package:googleapis_auth/auth_io.dart' as auth;
import 'package:http/http.dart' as http;
import 'package:gcloud/db.dart';
import 'package:gcloud/storage.dart';
import 'package:gcloud/pubsub.dart';
import 'package:gcloud/service_scope.dart' as ss;
import 'package:gcloud/datastore.dart' as datastore;
copied to clipboard
Getting access to the APIs #
The first step in using the APIs is to get an authenticated HTTP client and
with that create API class instances for accessing the different APIs. The
code below assumes that you have a Google Cloud Project called my-project
with credentials for a service account from that project stored in the file
my-project.json.
// Read the service account credentials from the file.
var jsonCredentials = new File('my-project.json').readAsStringSync();
var credentials = new auth.ServiceAccountCredentials.fromJson(jsonCredentials);

// Get an HTTP authenticated client using the service account credentials.
var scopes = []
..addAll(datastore.Datastore.Scopes)
..addAll(Storage.SCOPES)
..addAll(PubSub.SCOPES);
var client = await auth.clientViaServiceAccount(credentials, scopes);

// Instantiate objects to access Cloud Datastore, Cloud Storage
// and Cloud Pub/Sub APIs.
var db = new DatastoreDB(
new datastore.Datastore(client, 's~my-project'));
var storage = new Storage(client, 'my-project');
var pubsub = new PubSub(client, 'my-project');
copied to clipboard
All the APIs in this package supports the use of 'service scopes'. Service
scopes are described in details below.
ss.fork(() {
// register the services in the new service scope.
registerDbService(db);
registerStorageService(storage);
registerPubSubService(pubsub);

// Run application using these services.
});
copied to clipboard
The services registered with the service scope can now be reached from within
all the code running in the same service scope using the below getters.
dbService.
storageService.
pubsubService.
copied to clipboard
This way it is not necessary to pass the service objects around in your code.
Use with App Engine #
The gcloud package is also integrated in the Dart appengine package. This
means the gcloud services are available both via the appengine context and
service scopes. The authentication required to access the Google Cloud Platform
services is handled automatically.
This means that getting to the App Engine Datastore can be through either
the App Engine context
var db = context.services.db;
copied to clipboard
or just using the service scope registration.
var db = dbService;
copied to clipboard
Cloud Datastore #
Google Cloud Datastore provide a NoSQL, schemaless database for storing
non-relational data. See the product page
https://cloud.google.com/datastore/ for more information.
The Cloud Datastore API provides a mapping of Dart objects to entities stored
in the Datastore. The following example shows how to annotate a class to
make it possible to store instances of it in the Datastore.
@db.Kind()
class Person extends db.Model {
@db.StringProperty()
String name;

@db.IntProperty()
int age;
}
copied to clipboard
The Kind annotation tell that instances of this class can be stored. The
class must also inherit from Model. Now to store an object into the
Datastore create an instance and use the commit function.
var person = new Person()
..name = ''
..age = 42;
await db.commit(inserts: [person]);
copied to clipboard
The function query is used to build a Query object which can be run to
perform the query.
var persons = (await db.query<Person>().run()).toList();
copied to clipboard
To fetch one or multiple existing entities, use lookup.
var key = new Person()
..name = 'UniqueName'
..age = 42
..parentKey = db.emptyKey;
var person = (await db.lookup<Person>([key])).single;
var people = await db.lookup<Person>([key1, key2]);
copied to clipboard
NOTE: This package include a lower level API provided through the class
Datastore on top of which the DatastoreDB API is build. The main reason
for this additional API level is to bridge the gap between the different APIs
exposed inside App Engine and through the public REST API. We reserve the
rights to modify and maybe even remove this additional layer at any time.
Cloud Storage #
Google Cloud Storage provide a highly available object store (aka BLOB
store). See the product page https://cloud.google.com/storage/
for more information.
In Cloud Storage the objects (BLOBs) are organized in buckets. Each bucket
has a name in a global namespace. The following code creates a new bucket
named my-bucket and writes the content of the file my-file.txt to the
object named my-object.
var bucket = await storage.createBucket('my-bucket');
new File('my-file.txt').openRead().pipe(bucket.write('my-object'));
copied to clipboard
The following code will read back the object.
bucket.read('my-object').pipe(new File('my-file-copy.txt').openWrite());
copied to clipboard
Cloud Pub/Sub #
Google Cloud Pub/Sub provides many-to-many, asynchronous messaging. See the
product page https://cloud.google.com/pubsub/ for more information.
Cloud Pub/Sub uses two concepts for messaging. Topics are used if you want
to send messages and subscriptions are used to subscribe to topics and
receive the messages. This decouples the producer of a message from the
consumer of a message.
The following code creates a topic and sends a simple test message:
var topic = await pubsub.createTopic('my-topic');
await topic.publishString('Hello, world!')
copied to clipboard
With the following code a subscription is created on the topic and
a message is pulled using the subscription. A received message must be
acknowledged when the consumer has processed it.
var subscription =
await pubsub.createSubscription('my-subscription', 'my-topic');
var pullEvent = await subscription.pull();
print(pullEvent.message.asString);
await pullEvent.acknowledge();
copied to clipboard
It is also possible to receive messages using push events instead of pulling
from the subscription. To do this the subscription should be configured as a
push subscription with an HTTP endpoint.
await pubsub.createSubscription(
'my-subscription',
'my-topic',
endpoint: Uri.parse('https://server.example.com/push'));
copied to clipboard
With this subscription all messages will be send to the URL provided in the
endpoint argument. The server needs to acknowledge the reception of the
message with a 200 OK reply.
Running tests #
If you want to run the end-to-end tests, a Google Cloud project is required.
When running these tests the following environment variables need to be set:
GCLOUD_E2E_TEST_PROJECT
copied to clipboard
The value of the environment variable GCLOUD_E2E_TEST_PROJECT is the name
of the Google Cloud project to use. Authentication for testing uses
Application Default Credentials locally you can provide
GOOGLE_APPLICATION_CREDENTIALS or use
gcloud auth application-default login.
You will also need to create indexes as follows:
gcloud --project "$GCLOUD_E2E_TEST_PROJECT" datastore indexes create test/index.yaml
copied to clipboard