Chris Smothers

Early users may have noticed logging out a database entity, our primary datatype, resulted in output like this. 

![console.log(entity) before custom formatters](//images.ctfassets.net/3s1s2vyeq3pi/1k44LWN9bU51NARMLTfCbx/32ccb534ba6db81bddac351b102b4f9d/Screen_Shot_2021-04-06_at_10.19.40_AM.png)

Not what you'd hope for when trying to inspect your data. 

This is the result of bringing non-native lazy data structures to JSON.

With the release of [homebase-react 0.5.4](https://github.com/homebaseio/homebase-react/releases/tag/v0.5.4) we address this problem with custom chrome formatters.

![console.log(entity) after custom chrome formatters](//images.ctfassets.net/3s1s2vyeq3pi/6PcmsVfPpdDxSpHGXC7XrZ/94f5782ce109a744094ceb9ee34daaff/Screen_Shot_2021-04-06_at_10.33.04_AM.png)

Custom formatters are an experimental feature of chrome that have not yet been reified into the core docs, instead living in [this google doc](https://docs.google.com/document/d/1FTascZXT9cxfetuPRT2eXPQKXui4nWFivUnS_335T3U/preview "Chrome formatters documentation"). Despite their obscurity they are incredibly powerful and provide an interface to evaluate code when rendering and expanding logs.

To understand how homebase-react arrived at implementing custom formatters I need to explain a bit of our history. 

> If you're not interested in the backstory you can skip to [enabling custom formatters](#enabling-custom-formatters) so you can experience them yourself.

## Javascript vs. ClojureScript

Homebase React uses ClojureScript and its corresponding data format EDN internally. We then compile all of that to Javascript using the Google Closure Compiler (closure not Clojure) to get as small a bundle as possible. Yes, it's not as small as other state managers, but it's also an entire RDF style Datalog graph database and we think that tradeoff is worth it, especially for write-heavy applications and applications working towards being [local-first](https://news.ycombinator.com/item?id=21581444). Finally we provide APIs (react hooks) that accept JSON and do all the conversion to EDN and back again behind the scenes.

Our hope is to bring some of the benefits of these technologies to a wider audience, without everyone having to learn an entirely new paradigm or port a bunch of code.

In particular, EDN and Clojure provide far more safety and extensibility than JSON and Javascript; Clojure being immutable by default and EDN being extensible. This lets us build and support features that would be unwieldy in JSON and JS while still exposing most of the benefits with familiar JSON and JS idioms.

> See [Out of the Tar Pit](https://github.com/papers-we-love/papers-we-love/blob/master/design/out-of-the-tar-pit.pdf) for a more thorough justification of the benefit of immutability and functional purity in maintaining complex systems.

Should everyone give a functional immutable lisp a try once in their life? [We think so](http://clojurescriptkoans.com/ "Learn ClojureScript online with ClojureScript Koans"), but we're also pragmatic. We realize that the barriers to entry are high and the benefits are often unclear if you haven't already experienced them. Giving more developers an experience of just a few of those benefits is a big reason why homebase-react exists.

However, there are clearly some tradeoffs.

## Tradeoffs

1. **A larger bundle size.** Some of the Clojure runtime cannot be compiled away even though the closure compiler is really aggressive. In particular the implementation of immutable data structures and the functions that operate on them. In many ways adding CLJS is like adding Immutable JS with an even larger standard library of "immutable" functions. In fact, Immutable JS implements the same [persistent tree-shaped data structures](https://hackernoon.com/how-immutable-data-structures-e-g-immutable-js-are-optimized-using-structural-sharing-e4424a866d56) as Clojure. We also add EDN with its first class support for keywords, symbols, sets and custom data readers that go far beyond the expressivity of JSON and realize a lot of the promise of [JSON-LD](https://json-ld.org/), but without feeling shoehorned in.
2. **Clojure error messages sometimes leak into JS land.** We try to annotate the ones we know about so they make sense to JS devs, and we'll tell you exactly how to fix problems when we can, but it's far from perfect and if you see something weird please [create an issue](https://github.com/homebaseio/homebase-react/issues).
3. **Our code is released already minified.** We do this because most people do not develop with the [google closure compiler](https://developers.google.com/closure/compiler) and other build tools are not nearly as effective at optimizing ClojureScript code. This makes debugging homebase-react while developing a bit harder since the code is not very readable, but we think the tradeoff is worth it to provide a smaller bundle size. To ameliorate this problem we are building devtools like these formatters so you don't need to read the compiled source. We could compile away everything *but* the names and leave minification to library consumers, however this would balloon the bundle size on NPM even more and we've yet to see 'legibility of the compiled source' be a root problem for any homebase-react users.
4. **Confusing console logs.** *AKA why I'm writing this blog.* EDN data looks different from JSON data and to add to that, homebase-react mostly outputs entities, which are lazy datatypes and not very helpful when logged out with the default console formatting. So to summarize, EDN already looks foreign and now we're logging out the compiled and minified EDN of a datatype that's essentially just a pointer(db cursor) and does not even contain any interesting data.

## Enabling Custom Formatters
So, how do you turn them on? It's not hard, but there are some gotchas to be aware of.

If you develop with [Chrome](https://www.google.com/chrome/) or a Chromium browser like Brave or Edge you're in luck! You can enable custom formatters and you'll get significantly more meaningful logs for entities.

```jsx
const [myEntity] = useEntity(1)
console.log(myEntity)

// or 

const [myEntities] = useQuery({ item: { name: '$any' } })
console.log(myEntities)
```

These custom formatters allow us to perform lazy database queries to fetch all of an entity's attributes, including references to other entities **and** all reverse references to the current entity. They let you access your entire data graph from the console, with any entity you log out as an entry point.

**Steps to enable custom formatters**

**1.** Open the preferences panel in chrome devtools by clicking the cog.

![enable chrome formatters 1](//images.ctfassets.net/3s1s2vyeq3pi/7IB8zXRnjgzhyp5BHRgf4R/2cddb50afd5581b610ee689085b735d2/enable_chrome_formatters_1.png)

**2.** Toggle `Enabled custom formatters` on.

![enable chrome formatters 2](//images.ctfassets.net/3s1s2vyeq3pi/Ne8WIFDsUJ4GoBkomSKQd/98f94c511e280e9b5ed3962f72855df6/enable_chrome_formatters_2.png)

**3.** Keep the chrome console open and refresh the page. Any logged out entities should now have the custom formatting.

![enable chrome formatters 3](//images.ctfassets.net/3s1s2vyeq3pi/4k40iKocnIKV5g5vmTLgAX/b7ae932417720c7a5448c15c05293f92/enable_chrome_formatters_3.png)

**Live demo:** open the console while on the [todo example](https://homebaseio.github.io/homebase-react/#!/dev.example.todo) page.

**Remember**: for custom formatters to work `console.log(anEntity)` must be called **after** you open the chrome console. Anything logged out before you open the console will not have custom formatting applied because chrome processes those logs in the background.

### Why do attribute names look different?

The chrome formatters log out the raw underlying EDN, not JSON. Maybe one day we'll add the option to have the output transformed into JSON before it's rendered, but we think reading EDN is pretty intuitive even if you don't know it and we wanted to get this feature out quickly.

It's also a good way to get more familiar with the underlying data structures so when you do want to write more sophisticated queries that are not supported by our simplified JSON API, you won't be scared off by all the colons or lack of commas.

### So how do I read this data?

The only thing you really need to understand is `keywords` and how they map to JSON.

A keyword is a Clojure datatype primarily used for attribute names in key value style data structures. It has the format `:some.arbitrary.namespace/some-name`. So it starts with a `:` and then has an `optional namespace/` and a `name`.

```clojure
:email
:user/email
:io.homebase.user/email
```

So doing this in homebase-react

```jsx
transact({ item: { id: 42, name: "Deep Thought", motto: "DON'T PANIC" } })
``` 

Looks like this in EDN

```clojure
{:db/id 42
 :item/name "Deep Thought"
 :item/motto "DON'T PANIC"}
```

The word `item` is not represented internally as the name of a table or even a key to a `{ id: 1, name: "Deep Thought", motto: "DON'T PANIC" }` value. It's more flexible than that. It's basically RDF.

`item` is just the namespace prepended to each attribute other than `:db/id`. `:db/id` being a required attribute on all entities.

That's about it. You can now read EDN keywords and map them back to the JSON that created them.

### What is a reverse reference?

All `ref` attributes in homebase-react are directed, but can be queried and traversed bi-directionally. This is denoted by the `_` in attributes like `:todo/_owner` (see the screenshot above). This is a reverse ref; a set of all entities that have a `:todo/owner` ref to this "Stella" entity.

From the other side you can see that the "Fix Ship" entity does in fact have a `:todo/owner` ref pointing to "Stella".

#### `_recentlyTouchedAttributes`

**DEPRECATED**

Our first attempt to make entities less opaque took the form of `entity._recentlyTouchedAttributes`. An object containing all the the recently queried attributes of an entity that we'd tack onto entities when you set `<HomebaseProvider config={{ debug: true }}>`. It was far from what anyone wanted and is now deprecated with the introduction of custom formatters.

## cljs-devtools

None of this would have been possible without a project that we rely on internally and that forms the foundation for our homebase-react custom formatters [cljs-devtools](https://github.com/binaryage/cljs-devtools). This is one of the first and undoubtedly one of the most sophisticated uses of chrome formatters. Without this project custom formatters would likely have been [cut from chromium](https://github.com/binaryage/cljs-devtools/issues/55#issuecomment-632384007) a while ago.

## EDN

I also want to shoutout EDN. It's an incredible serialization format and the primary reason we're willing to jump through all these hoops and build any of this. I'm sure we'll write more about it in coming posts, but in the meantime, if you're intrigued, [the spec](https://github.com/edn-format/edn) is incredibly friendly and section `3.1.2` and `3.1.1` of [A History of Clojure](https://download.clojure.org/papers/clojure-hopl-iv-final.pdf) cover the rationale behind it.

---

Thanks for reading all the way here! I hope you enjoy these formatters and please keep letting us know where the sticking points are.

hero

Better debugging through custom chrome formatters with database access in the console.

cover

Homebase React 0.5.4 | Custom Chrome Console Formatters

Member of the #RoamCult, notetaking nerd, or ready to build your own second brain application? We built an open source Roam Research clone in a week using Homebase React. 

Find and fork our demo [here](https://github.com/homebaseio/homebase-react/tree/master/examples/roam).

![Screen Recording 2021-02-25 at 9.47.25 AM](//images.ctfassets.net/3s1s2vyeq3pi/7LgZzTiHKPiN3qRtPmMtBf/0e9835cda70293e379d1be354e78d85c/Screen_Recording_2021-02-25_at_9.47.25_AM.gif)
*Our Roam Clone in action with Hacker News data.*

roam-cover

We built an open source Roam Research clone in a week using Homebase React.

Build Your Own Roam Clone in Homebase React

[Homebase React](https://github.com/homebaseio/homebase-react) v0.5.0 simplifies relationships by automatically normalizing nested JSON transactions.

Now you can write

```jsx
transact([
  { todo: { name: 'T1', project: { name: 'P1' } }
])
```

Which is equivalent to

```jsx
transact([
  { project: { id: -1, name: 'P1' }, 
  { todo: { name: 'T1', project: -1} }
])
```

In addition to simplifying the API for transactions we’ve also simplified schema maintenance. 

Homebase React requires a schema on read to support relationships. Basically, instead of joining data at query time like in SQL, you specify how attributes are related up front so queries can be simpler.  

With nested JSON transaction we can detect relationships at transaction time. This allows us to prompt you to add the correct schema if it’s missing. So even if you’re new to normalized data we’ll guide you to some reasonable defaults.

```jsx
const config = {}

<HomebaseProvider config={config} />

transact([{ item: { child: { name: 'A' }}}])
// Throws =>
`
Error: The 'item.child' attribute should be a ref type of one.

Add this to your config:  schema: { item: { child: { type: 'ref', cardinality: 'one' }}}
`
```

And of course you can easily traverse your graph with `entity.get`

```jsx
const [t1] = useEntity({ todo: { name: 'T1' } })
t1.get('project', 'name') // => 'P1'
```

## Arrays

In addition to nested JSON objects we’ve added support for one level of nested arrays. It’s not full JSON support, no arrays of arrays `[[1]]`, but it’s pretty close. 

```jsx
transact([{ 
  project: { 
    name: 'P1', 
    numbers: [1, 2, 3],
    todos: [
      { todo: { name: 'T1' } },
      { todo: { name: 'T2' } }
    ]
  } 
}])
```

To access array elements we’ve added several helpful features to the get API.

### Get array index

```jsx
const [p1] = useEntity({ project: { name: 'P1' } })
p1.get('numbers', 0, 'value') // => 1
p1.get('todos', 0, 'ref', 'name') // => 'T1'
```

### Implicit map over attribute

Omitting an index on an array type will automatically cause the next attributes to be mapped over all items in the array.

```jsx
const [p1] = useEntity({ project: { name: 'P1' } })
p1.get('numbers', 'value') // => [1, 2, 3]
p1.get('todos', 'ref', 'name') // => ['T1', 'T2']
```

### Array attributes

You probably noticed the the attributes `'value'` and `'ref'`. These are special attributes on arrays. If an array element is a value like a string or a number you can access it with the value attribute. If an array element is a reference to a related entity then you can get that entity with the ref attribute.

### Array order

Arrays also have a special `'order'` attribute that contains a number representing the sort order of an array element. You can update this attribute to reorder an array.

```jsx
const [p1] = useEntity({ project: { name: 'P1' } })
p1.get('numbers', 'value') // => [1, 2, 3]
p1.get('numbers', 0, 'order') // => 1
p1.get('numbers', 1, 'order') // => 2
transact({ 'homebase.array': {
  id: p1.get('numbers', 2, 'id'),
  order: 1.5
}})
p1.get('numbers', 'value') // => [1, 3, 2]
```


Homebase React v0.5.0 simplifies relationships by automatically normalizing nested JSON transactions.

cover@4x

Homebase React 0.5.0 | JSON Derived Relationships

At Homebase, we think **a lot** about [client-side data](https://github.com/homebaseio/homebase-react). One problem we wonder about is the complexity of state management libraries. Redux was a huge step forward when it came on the scene. It forced us to think about how we mutate and subscribe to changes in our data along with giving us places to put that code. Over time, however, we realized we might have too many places to put that code... 

> I don’t like Redux very much, but this sounds like the opposite of its problems? With Redux, you *do* know deterministically where the code lies because you can follow the reducer chain. The problem is that what should’ve been local often becomes global.
> 
> — Dan Abramov (@dan_abramov) [March 22, 2020](https://twitter.com/dan_abramov/status/1241756566048694272)

*Even the co-creator of Redux, Dan Abramov, sounds ready for the next stage in state management.*

Luckily for us, over the past few years programmers have created several new approaches to state management. In this post, we share our research on the trends in post-Redux state management and how we tried to incorporate the best parts of existing libraries into one solution. We hope you find a new library to play with and maybe even use in production.

## Trends

We went through the most popular state management libraries and analyzed what developers wanted to see in their state management libraries. Below are some trends we got excited about:

### Simple API

Ever since Redux, we and developers we've spoken to craved a simpler way to handle state. It makes sense that most state management alternatives we looked at prioritized simplicity. [Zustand](https://github.com/pmndrs/zustand) really shines here, seeming to prioritize a simple API above all else.

```jsx
import create from 'zustand'

const useStore = create(set => ({
  bears: 0,
  increasePopulation: () => set(state => ({ bears: state.bears + 1 })),
  removeAllBears: () => set({ bears: 0 })
}))
```

*A code example from Zustand.*

### Declarative

Another way to make code simpler is to make it [declarative](https://en.wikipedia.org/wiki/Declarative_programming). This is the main selling point of [Cerebral](https://cerebraljs.com). While we are huge fans of simplicity through declarative programming, we found Cerebral more complicated than we'd like.

```jsx
import * as React from 'react'
import { connect } from '@cerebral/react'
import { state, sequences } from 'cerebral'

export default connect(
  {
    inputValue: state`inputValue`,
    changeInputValue: sequences`changeInputValue`
  },
  class MyComponent extends React.Component {
    componentDidMount() {
      this.props.reaction(
        'focusUsername',
        {
          error: state`usernameError`
        },
        ({ error }) => error && this.input.focus()
      )
    }
    render() {
      return (
        <input
          ref={(node) => {
            this.input = node
          }}
          value={this.props.inputValue}
          onChange={(event) =>
            this.props.changeInputValue({ value: event.target.value })
          }
        />
      )
    }
  }
)
```

*A code example from Cerebral.*

### Hooks

React hooks are another path to simplicity since they eliminate the amount of concepts you need to remember when writing React applications. With the advent of React Hooks, we saw libraries  such as [Zustand](https://github.com/pmndrs/zustand) and [Unstated](https://github.com/jamiebuilds/unstated) start to use hooks for state management.

```jsx
function useCounter(initialState = 0) {
  let [count, setCount] = useState(initialState)
  let decrement = () => setCount(count - 1)
  let increment = () => setCount(count + 1)
  return { count, decrement, increment }
}

let Counter = createContainer(useCounter)

function CounterDisplay() {
  let counter = Counter.useContainer()
  return (
    <div>
      <button onClick={counter.decrement}>-</button>
      <span>{counter.count}</span>
      <button onClick={counter.increment}>+</button>
    </div>
  )
}
```

*A code example from Unstated.*

### Server State

One of the developments we're most excited about in the state management space is the acknowledgement of "server state". Both [Recoil](https://recoiljs.org/docs/guides/asynchronous-data-queries/) and [React Query](https://react-query.tanstack.com/overview) try to make it easy to integrate client state with your server state using Asynchronous data queries. The trade-off ends up being added complexity.

```jsx
const currentUserNameQuery = selector({
  key: 'CurrentUserName',
  get: async ({get}) => {
    const response = await myDBQuery({
      userID: get(currentUserIDState),
    });
    return response.name;
  },
});

function CurrentUserInfo() {
  const userName = useRecoilValue(currentUserNameQuery);
  return <div>{userName}</div>;
}
```

*A code example from Recoil.*

## Putting it all together

Looking at these trends we are more excited than ever about front-end development and decided to throw our hat in the ring with a state management library of our own. We wanted the simplicity of a library like Zustand with the power of a library like Recoil. 

So, in a month we ended up creating a state management library around the local-first database we are working on and voila: very simple, imperative code, but with the functionality of a relational graph database right at your finger tips in the client.

As for "server state", we are working on automatic syncing client and server data via a [CRDT](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type).

It's already integrated into several applications, but we'd love to hear your thoughts on [Homebase React](https://github.com/homebaseio/homebase-react).

```jsx
const Counter = () => {
  const [counter] = useEntity({ identity: 'counter' })
  const [transact] = useTransact()
  return (
    <div>
      Count: {counter.get('count')}
      <div>
        <button onClick={() => transact([{
          counter: {
            id: counter.get('id'),
            count: counter.get('count') + 1
          }
        }])}>
          Increment
        </button>
      </div>
    </div>
  )
}
```

*A code example from Homebase React.*

## Conclusion

We hope this was helpful! The chart below summarizes what we discussed and does a breakdown of what features each library has prioritized.

### State management feature comparison

|Library    |Client State|Declarative|Server state|Simpler|Hooks|
|-----------|:------------:|:-----------:|:------------:|:--------------------:|:-----:|
|Cerebral   |✓           |✓          |            |                    |     |
|Zustand    |✓           |           |            |✓                   |✓    |
|Unstated   |✓           |           |            |✓                   |     |
|Recoil     |✓           |           |✓           |                    |     |
|React Query|            |           |✓           |                    |     |
|Redux      |✓           |           |            |                    |✓       |
|Homebase   |✓           |✓          |✓           |✓                   |✓    |
|MobX       |            |           |            |✓                   |     |
|Overmind   |            |           |            |✓                   |     |
|Freactal   |            |           |            |✓                   |     |

#### Notes: 
* Redux does have middleware to try to incorporate server state.
* Unstated has unreleased React hooks.


We share our research on the trends in post-Redux state management and how we tried to incorporate the best parts of existing libraries into one solution.

Blog

Homebase React 0.5.4 | Custom Chrome Console Formatters

Build Your Own Roam Clone in Homebase React

Homebase React 0.5.0 | JSON Derived Relationships

State Management Beyond Redux