Datomic Pro and Datomic Cloud are FREE!

Datomic Lookup Refs

13 January 2014

Datomic's new lookup refs provide an easy way to identify entities by domain unique identifiers.

Rationale

Datomic supports several approaches to identity, including internal identifiers (entity IDs that are unique within a database) and external identifiers (domain attributes that are designated to be :db.unique/identity).  Lookup refs provide a concise data representation that accords to external ids the powers of internal ids, and can be used interchangeably with internal ids in the Datomic API. This is powerful, since often, e.g. in the front-end of a system or during imports, you already have the external ids in hand. With lookups refs you can often avoid having to incorporate a lookup in application code since the lookup will be done implicitly when resolving the lookup ref (thus the name).

Implementation

A lookup ref is simply a java.util.List containing two elements: a unique attribute and a value for that attribute.  Encoded in edn, a Lookup ref looks like this:

[:org/email "info@datomic.com"]

Lookup refs can be used in the entity API, or in any of the database index APIs.  For example, the following code retrieves an entity:

// Groovy
db.entity([':org/email', 'info@datomic.com'])
;; Clojure
(d/entity db [:org/email "info@datomic.com"])

Lookup refs can also be used to refer to existing entities in transaction data.  This allows updating transactions to specify :db/id directly instead of using a dummy upsert, as shown below:

;; edn transaction data, using temp :db/id and dummy upsert
[{:db/id #db/id[:db.part/user]
 :org/email "info@datomic.com",
 :org/favoriteColor :blue}]

;; edn transaction data, using lookup ref
[{:db/id [:org/email "info@datomic.com"],
 :org/favoriteColor :blue}]

Similarly, they can be used in transactions to build ref relationships.

We expect that lookup refs will expedite a wide variety of use cases, including imports, interactive exploration, and client application development.  

We look forward to your feedback.

..

Schema Alteration

23 December 2013

Datomic is a database that has flexible, minimal schema. Starting with version 0.9.4470, available here, we have added the ability to alter existing schema attributes after they are first defined. You can alter schema to

  • rename attributes
  • rename your own programmatic identities (uses of :db/ident)
  • add or remove indexes 
  • add or remove uniqueness constraints
  • change attribute cardinality
  • change whether history is retained for an attribute
  • change whether an attribute is treated as a component

Schema alterations use the same transaction API as all other transactions, just as schema installation does.  All schema alterations can be performed while a database is online, without requiring database downtime.  Most schema changes are effective immediately, at the end of the transaction.  There is one exception: adding an index requires a background job to build the new index. You can use the new syncSchema API for detecting when a schema change is available.

When renaming an attribute or identity, you can continue to use the old name as long as you haven't repurposed it. This allows for incremental application updating.

See the schema alteration docs for the details.

Schema alteration has been our most requested enhancement. We hope you find it useful and look forward to your feedback.

..

Datomic 2013 Recap

08 December 2013

2013 was a great year for Datomic.  The value of a flexible information model and immutable data have proven themselves time and again.  Customers have built a variety of powerful systems, taking advantage of

  • ACID transactions
  • pluggable SQL/NoSQL/cloud storage
  • complete access to the history of information
  • the Datalog query language
  • elastic read scalability
  • a granular information model
Over the course of the year, we produced over 40 Datomic releases. The API has been remarkably stable: Our commitment to a strong architecture has allowed us to focus on adding features and fleshing out the vision, without the churn of revisiting past decisions.  

A major new feature is the Datomic Console, a graphical UI for exploring Datomic databases.  The console provides a great visual introduction to the Datomic information model.  It supports exploring schema, building and executing queries, navigating entities, examining transaction history, and walking raw indexes. 

We made several API additions:
  • Excision, a sound model (and API) for permanent removal of data, with auditability.
  • The log API provides the ability to access the log, which is more properly viewed as a time index.
  • The seekDatoms and entidAt APIs provide advanced capability for accessing Datomic's indexes, augmenting the datoms API.
  • The sync API allows multiple processes to coordinate around points in time-of-record, or relative to local process time.
  • Transaction map expansion automates the creation of arbitrarily nested data.
We also made a number of operational improvements:
  • We added Cassandra and to the list of supported storage, in addition to the existing options of DynamoDB, SQL, filesystem, CouchBase, Infinispan, and Riak.
  • The Starter Edition of the Datomic Pro license makes all storages available, for free.
  • We have added a number of new CloudWatch metrics, and a pluggable metrics API for integration with other systems.
  • The MusicBrainz sample database is a great dataset for exploring Datomic.
  • We continue to track AWS best practices, now supporting IAM roles for distributing credentials and DynamoDB local for testing.
We are looking forward to an equally exciting 2014. We will be delivering a number of new features requested by users, plus a few big surprises.

Many thanks to our customers and early adopters for your support and feedback.

Happy New Year!
..

Cassandra Support

18 November 2013

We are pleased to announce alpha support for Cassandra as a storage service for Datomic, now available in version 0.9.4384.

Cassandra is an elastically scalable, distributed, redundant and highly-available column store. Recent versions of Cassandra added support for compare-and-swap operations via lightweight
transactions using an implementation of the Paxos protocol. Datomic leverages this mechanism to manage a small number of keys per database that require coordinated access, while the bulk of a database's content is written as immutable data to a quorum of replicas in the cluster.

Cassandra support requires a minimum Apache Cassandra 2.0.2 or newer, or equivalent. Native CQL protocol support must be enabled. Cross-data center deployments are not supported. Cassandra internal security is supported, but optional.

This release represents preliminary support based on requests from users. We are very interested in feedback.

For instructions on configuring Cassandra for use with Datomic, see Setting Up Storage Services.


..

Using IAM Roles with Datomic on AWS

25 October 2013

With today's Datomic release, you can use IAM roles to manage permissions when running in AWS.

Motivation

Datomic's AWS support has been designed according to the principle of least privilege.  When running in AWS, a Datomic transactor or peer needs only the minimum permissions necessary to communicate with various AWS services.  These permissions are documented in Setting Up Storage Services.

But you still need some way to install these minimal permissions on ephemeral virtual hardware. Early versions of AWS left this problem to the developer.  Solutions were tedious and ad hoc, but more important they were risky.  Leaving every application developer the task of passing credentials around is a recipe for credentials lying around in a hundred different places (or even checked into source code repositories.)

IAM roles provide a generic solution to this problem.  From the FAQ: "An IAM role allows you to delegate access, with defined permissions, to trusted entities without having to share long term access keys"  (emphasis added).  From a developer perspective, IAM roles get credentials out of your application code.

Implementation

Starting with version 0.9.4314, Datomic supports IAM roles as the default mechanism for conveying credentials in AWS.  What does this mean for developers?
  1. If you are configuring Datomic for the first time, the setup instructions will secure peers and transactors using IAM roles. 
  2. If you have an existing Datomic installation and want to upgrade to roles, Migrating to IAM Roles will walk you through the process.
  3. Using explicit credentials in transactor properties and in connection URIs is deprecated, but will continue to work.  Your existing deployments will not break.
IAM roles make your application both easier to manage and more secure.  Use them.
..

9 of 13
Datomic Pro
Benefits Documentation Release Notes Apache License
Datomic Cloud
Product Details Support Pricing Documentation Release Notes Apache License
Resources
Blog Knowledgbase Developer Forum Product Support
Company
About UsContact
©2016-2021 Cognitect, Inc.
Datomic® is a registered trademark of Cognitect, Inc.
Trademark Guidelines | Privacy Policy