Monday, September 17, 2012

Datomic Monitoring and Performance

I have just added a new section to the Datomic documentation on monitoring and performance. If you are tuning Datomic, it is a must-read.

I want to call attention in this blog post to one aspect of Datomic monitoring that might not be immediately apparent. Which is that you can use Amazon CloudWatch to monitor any instance of Datomic Pro, regardless of which storage you are using, and regardless of whether you are running any of your processes in the AWS cloud.

Part of the value of a good cloud architecture is being able to mix-and-match the pieces.

Thursday, September 6, 2012

REST API

I'm pleased to announce that, starting with version 0.8.3488, Datomic now offers a REST API.

There are a number of reasons to do this, first and foremost is that it will now be possible to access Datomic from non-JVM languages.

How does it work?

The command bin/rest will start up a peer which runs as a stand-alone HTTP server. You can access that service from any application using any language and any HTTP library. That's it!


This greatly enhances your architectural options:


Q&A


So, does this make Datomic client-server? 

Yes and no. First off, the 'servers' are themselves peers, so you still get elastic, configuration-free horizontal query scalability by simply starting/stopping more peers. Second, the Datomic model, which substantially mitigates the problems of client-server databases, is faithfully replicated for clients:

  • A peer service can serve more than one database, against more than one storage (Pro)
  • Clients get the same kind of data-driven API as do peers
  • Clients can get repeatable queries, and multiple queries with the same basis
  • Clients get the time-travel as-of and since capabilities, raw access to indexes and ranges
  • Clients can issue multi-datasource queries, and pass data to query
  • Clients can get a push feed of transaction events, via a Server-Sent Events source

Basically, it's just more options. Build full peers in a JVM language or lightweight clients in any language, or any combination.

Do these clients count towards my peer count? 

No. The clients do not run the Datomic peer library, and thus do not count. Each REST service process counts as a peer.

Does this work with Datomic Free? 

Yes it does.

What's the status of the API? 

It's still alpha, as we want to incorporate your feedback.

Are there client libraries for language _______?

Not yet. This is something we hope each language community will help us build.

Where can I get more details?

Check out the REST API docs


We hope you enjoy this new API and the capabilities it affords. As always, we welcome your feedback and input. If you've been on the sidelines waiting for Datomic to come to your favorite language - welcome!

Monday, September 3, 2012

ElastiCache in 5 minutes

You can add ElastiCache to your AWS-based Datomic system in minutes:
  • Login to your ElastiCache management console and choose "Launch Cache Cluster"
  • Complete the wizard. You can accept all defaults.
  • Back at the console, select "Cache Security Groups":
  • ... and authorize your Datomic system's security group (default is "datomic"):

  • Back at the console, select "Cache Clusters" and copy the node endpoints string:

  • Paste the node endpoints into your transactor properties file:
    memcached=foo.use1.cache.amazonaws.com:11211
  • Set the node endpoints in your peer application code:
    System.setProperty
    ("datomic.memcachedServers",
     "
foo.use1.cache.amazonaws.com:11211);

That's all there is to it.  Datomic will transparently use ElastiCache. There is no need to configure any cache timeouts, or change any code. Datomic's key names will not conflict with any other use, so you can use the cache for other tasks as well.

You can also configure different caches for different process in the system. For example, I keep a local memcached process running at the office, so from a cold peer (my development laptop) queries are fast, even when I am connecting to a transactor that is running in the AWS cloud.