Wednesday, December 14, 2016

Customer Feedback Portal

As part of our commitment to improving Datomic, a few weeks ago we enabled a new feature request and user feedback system, Receptive.io, where you can help us prioritize our efforts and help shape the future of Datomic.

To submit your feature request follow the "Suggest Features" link in the top navigation of the my.datomic.com dashboard. We have already connected your account to Receptive.io so everything is set up and ready for you to go.

You can read more about using Receptive here.

-The Datomic Team

Monday, November 28, 2016

Datomic Update: Client API, Unlimited Peers, Enterprise Edition, and More

We are pleased to announce that the latest (0.9.5530) release of Datomic includes a set of new features and licensing changes to address needs identified by our customers:
  • In addition to the peer model, Datomic now includes a Client API suitable for smaller, short lived processes, e.g. microservices.
  • The various tiers of the Datomic Pro license model have been simplified to a single license with no restriction on peer count.
  • We have introduced an Enterprise license tier for users who need customized pricing, support, or licensing terms.
  • Tempids and explicit partitions are now optional, simplifying code for the many programs that do not care about them.
  • Schema install and update are now implicit, and do not require explicit :db.install/attribute or :db.update/attribute datoms.

The features described above are additive and opt-in, so take advantage of them as and when you please.

Each of these changes is described in more detail below.

Building On a Solid Foundation


Before talking about what is new, it is important to talk about what is unchanged. We built Datomic believing that the Rationale is a sound foundation for an information system, and experience has proven this out. We have not retracted a word of the rationale since day one, and are not doing so today. Datomic’s core ideas are unchanged:
  • getting time, process, and perception right
  • sound data model
  • ACID transactions
  • Datalog query
  • minimal schema
  • separate reads and writes
  • programming with data

Datomic has delivered these ideas with a discipline that minimizes breaking API change. As a result, Datomic users have been able to focus on their business problems without having to worry about changing semantics in their database.

Client API


Datomic’s peer library puts database query in your own application process. This provides several benefits, but at the price of a heavier dependency (both in code and in memory requirements) than a traditional client.

A smaller footprint is useful in environments that have operational limitations, or where processes are small or short-lived. The new Datomic client API addresses this need. Lightweight clients connect to Peer Servers, which are peers that run in a separate address space.

Existing peers are unchanged, and you can mix and match peer and client applications as you see fit within the same Datomic install. Clients and peers are described in detail in the new clients and peers section of the docs.

With today’s release, we are making available the alpha version of the open source Client library for Clojure. The Java library will be released shortly. We also have plans to both create more language libraries for Client and enable our customers to create their own. We are interested in your feedback on the Client API itself and the priority of our language reach efforts. As of today, we have enabled a customer feedback portal, accessible via the "Suggest Features" link in the top navigation of the my.datomic.com dashboard, where you can help us prioritize our efforts in this (and many other) areas.

Unlimited Peers


Flexibility in Peer use has been the most often-requested update to Datomic. You are solving complex problems using cutting edge technologies and architectures. Your tools should allow you to design the system that best fits your needs.  Datomic’s new licensing model gives all users - Starter, Pro and Enterprise - the ability to design for and deploy as many Peer processes (and Clients!) as their systems require. Today’s release represents a massive upgrade to the potential of each (new and existing) Datomic installation.

Pro Starter License


The Pro Starter license provides a no-cost way to try Datomic. You get a perpetual license plus a year of software upgrades for free. Starting with this release, Pro Starter includes all the features of a Pro license, including
  • unlimited peers
  • clients
  • High Availability (HA)
  • integrated memcached

Enterprise Tier


Datomic has a number of enterprise customers already. They distinguish themselves by wanting
  • custom license terms
  • custom pricing for larger installations
  • custom support terms
  • custom development

If you match one or more of these criteria, contact us to discuss an Enterprise license.

Tempid and Partition Defaults


Datomic’s tempids provide a way to partition new entities, encoding a locality hint directly in transactions. This feature is powerful, but rarely used, and the API and data structure for tempids are an inconvenience for the majority of users, who do not need or want partition control.

Starting with the current release of Datomic:
  • tempids are optional
  • when you need a tempid to coordinate the relationship between two entities, you can use an ordinary string instead of a tempid structure, and that string can be meaningful to readers of your code
  • the existing tempid data structure and API continue to be supported unchanged. Use them if you want them.

Clients will support string tempids only.

Schema Install and Update


Transactions that change attribute schema must include either :db.install/attribute (to create an attribute) or:db.alter/attribute (to change an existing attribute). The new release of Datomic infers the need for these datoms and adds them to your transaction automatically, reducing the verbosity of schema data.

Conclusion


We are very excited about the additions and changes to Datomic. To celebrate, we will be offering a 20% discount on new Datomic purchases through the end of February 2017. We hope you take advantage of the new features and this discount opportunity and please feel free to reach out to us at anytime.

Wednesday, August 10, 2016

Log API for Memory Databases

The most recent Datomic Release provides access to the Datomic Log API for memory databases. I would like to take this opportunity to describe some of the features and uses of the Datomic Log API.

The transactional log is a fundamental component of ACID database systems, a durable record of the transactions performed by the database. In addition to its critical function in ensuring ACID semantics, the Datomic log, as a sequential (in database time, t) record of all transactions, also functions as a time-ordered index to the data stored in a Datomic database.

Datomic provides access to the transaction log directly via the tx-range function and from within query using the tx-ids and tx-data functions.

An example using the log in query is now available in the Day of Datomic repo. Our example database records the streets on which our three protagonists, John, Mary, and Joe, live.

Let’s find out when Joe moved to Broadway:

(d/q '[:find ?tx
       :in $ ?name ?street
       :where
       [?e :person/name ?name]
       [?e :person/street ?street ?tx true]]
     (d/history (d/db conn)) "Joe" "Broadway")

This query returns 13194139534317, the transaction ID of the transaction that asserted Joe’s street is Broadway. As in all Datomic databases, every transaction also records a timestamp, the :db/txInstant. Let’s see what wall-clock time is associated with this transaction entity:

(d/pull (d/db conn) '[:db/txInstant] 13194139534317)

So Joe moved to Broadway in 1983.

Issuing queries against the transaction log is a powerful approach for auditing the operational history of a database. Because every transaction is an entity in Datomic, we can easily retrieve the entire set of datoms for a given transaction entity. Let’s find out what else happened in the ‘Joe moves to Broadway’ transaction. This query returns all the datoms associated with the given transaction:

(d/q '[:find ?e ?a ?v ?tx ?op
       :in ?log ?tx
       :where [(tx-data ?log ?tx)[[?e ?a ?v _ ?op]]]]
     (d/log conn) 13194139534317)

;; result:
#{[17592186045420 64 "2nd" 13194139534317 true]
  [17592186045419 64 "Broadway" 13194139534317 true]
  [13194139534317 50 #inst"1983-01-01T00:00:00.000-00:00" 13194139534317 true]
  [17592186045420 64 "Elm" 13194139534317 false]
  [17592186045419 64 "1st" 13194139534317 false]}

Note that we see the same wall clock time we found previously as well as 4 other datoms. One is the assertion of Joe moving to Broadway, one is the retraction of his previous street (1st), and the remaining two datoms are about someone else entirely. Let’s find out who:

(d/pull (d/db conn) '[*] 17592186045420)

;; result:
{:db/id 17592186045420, :person/name "Mary", :person/street "2nd"}

By using the log, we’ve determined that Mary’s move to 2nd Street was recorded at the same time (during the same transaction) as Joe's move to Broadway.

The ability to query the Datomic transaction log directly is a powerful tool for managing, administering, and using a Datomic database. The addition of the Log API to memory databases enables low-overhead testing and evaluation of Datomic’s Log API feature. We hope you find the Log API on memory databases a helpful addition for lightweight development and unit testing.

Saturday, April 23, 2016

You Might Not Need an ORM

Over the last few months, my colleague Michael Nygard has been writing The New Normal series over on the Cognitect blog, arguing that our industry needs to embrace continuous partial failure and aim to build antifragile systems.

In his most recent installment, Mike shares some ideas on sharp tools and their value in eliminating code. Mike worked with several teams that considered object-relational mapping (ORM) for Datomic, concluding that with Datomic, you don't need ORM at all. Datomic lets you eliminate an entire architectural layer from your applications.

Thursday, December 3, 2015

Reified Transactions

Domain data often answers the question "what", e.g.
  • What products are in inventory?
  • What users have accounts on the system?
  • What tests have been run?
But many domains also include information about the provenance of data, e.g.
  • Who said this data was correct?
  • When should this data be published?
  • Where did this data come from?
  • Why was this correction necessary?
How should you model this "data about data"? Datomic stores (and lets you store) facts about transactions themselves. In other words, transactions are themselves reified as data in the system.

Tim Ewald gave a great talk about this at Datomic Conf. His talk and all the other Datomic Conf talks are now freely available online.

Tuesday, November 24, 2015

CQRS with Clojure, Kafka, and Datomic

Datomic's clean separation of reads and writes has inspired many comparisons with the CQRS (Command and Query Responsibility Segregation) architectural pattern.

One person who has traveled far down this road is Bobby Calderwood, a member of the Technology Fellows team at Capital One. Check out his recent Clojure/conj talk, where he demonstrates a complete CQRS implementation using Clojure microservices, Kafka, and Datomic.

Thursday, November 5, 2015

Yeller chooses Datomic

Yeller is an exception tracking service whose goal is to provide clear, actionable information to developers to take the pain out of examining application errors. They chose Clojure and Datomic as the platform for Yeller in order to make the development process more effective and more efficient.

You can read all about their reasons for choosing Clojure and Datomic on their blog. You can also watch Tom explain it all at EuroClojure 2015.