Obsidian, Roam, and the rise of Integrated Thinking Environments

Published Nov 2, 2021

• Articles
• Integrated Thinking Environments
• Obsidian
• Knowledge Innovation
• Information Systems
• Data

In this article, I discuss:

• The (re-)emergence of a new class of apps called “Integrated Thinking Environments (ITEs)”;
• The NOTE framework, used to describe the specific feature sets ITEs provide users;
• Some examples of ITEs; and
• Some future directions for ITE use, design, and development.

The key takeaways:

• ITEs provide users with specific and customizable toolsets to be able to navigate, organize, and transform their thinking—and these toolsets are highly extensible.
• To use your thoughts, you capture notes—notes are instantiations of your thinking that adhere to your model of your world.
• Some key future directions in ITEs are…
  1. the development of customized, downloadable environments for specific uses,
  2. new AI and machine learning tools that support their functions,
  3. more use of ITEs in education and more education on ITEs,
  4. the advancement of techniques and skills deeply linked to the use of ITEs, and
  5. the advancement of ways of collaborating when using ITEs.

The “notes renaissance” we’re currently experiencing is being driven by an explosion of apps that are, in fact, Integrated Thinking Environments.

An Integrated Development Environment, or IDE, is an app used by programmers to develop software.

IDEs provide a comprehensive set of tools to make coding easier. For instance, an IDE might help you debug your code by identifying obvious errors or letting you test your code as you write it. It might show you definitions of the functions and variables you’re using so you can reference their documentation without leaving the coding context. Some IDEs can even do “intelligent code completion,” offering ways of finishing the keywords you’re inputting or suggesting the parameters that you should include in a new line of code.

In a way, IDEs provide a kind of augmented cognition for programmers. They reduce the cognitive capacity needed for software development by automating some of the work. This is freeing: it allows programmers to think less about coding, and more about design and development.

That’s the dream, isn’t it? Our tools should make our work easier, enabling us to be more innovative. So why do programmers get to have all the fun? Well, good news. I believe that the “notes renaissance” we’re currently experiencing is being driven by an explosion of apps that are, in fact, Integrated Thinking Environments.¹

An Integrated Thinking Environment (ITE) is an app that provides tools to make thinking easier, enabling us to be more innovative.² ITEs provide features that take care of some parts of the work of knowledge management and knowledge innovation. In the apps that I’ve seen, these features help us craft personal, contextual ITEs.

Some popular examples of ITEs include DEVONthink, Roam Research, Notion, and Obsidian.

An IDE is to code as an ITE is to… what? How do we represent our thoughts in these systems? How should we be structuring thoughts to use them in an ITE?

“Hold on a second,” you’re saying. “Those apps you just listed are all notes apps. Is an ITE just a notes app?”

Yes! ITEs are probably a subset of notes apps, just like IDEs are a subset of text editors.³ Every IDE is fundamentally a text editor. Every ITE is fundamentally a notes app. That’s because—at least for now—the best way we have of working with our thoughts is to instantiate them as notes.⁴

Instantiation is the representation of something in the real world in an information system. We should think about instantiating our thoughts in terms of our conceptual model of the world around us.⁵ What are we trying to represent? What’s important about it? What is it made of, how does it fit into the world, and what is its structure? When we take a note about a thing, we are asking these questions about representing the thing. Whatever our answers are decides what a note should look like. Put differently: when we take note of something, we are converting our thinking about that thing into unstructured or semistructured data.⁶

Remember, IDEs are for software development. Developers instantiate the design and functions of the apps we use as code in an IDE. This typically means putting text in certain syntaxes that the IDE (or a compiler) can parse in order to build the apps we use. In turn, some IDEs can even show what the results might look like and telling you if there are errors.

So, when we instantiate our ideas, arguments, and observations as notes in an ITE, we should also consider how we can best parse and compile these notes into useful things.

This is why we need more than a simple text editor for thought processing. Arguably, you can take notes with the most basic of text editing apps, such as NotePad (Windows) or TextEdit (macOS). In fact, combined with their operating systems, text editors also provide basic mechanisms for navigating and organizing whatever you write in them.

This new crop of note-taking apps, however, provide features that go beyond these basic mechanics. In the next section, I describe four dimensions of ITEs: Navigability, Organizability, Transformability, and Extensibility (NOTE).⁷ The affordances provided by these four dimensions—and the interactions between them—are what make these tools ITEs.

In my view the apps I’m calling ITEs go quite a bit further than a conventional text editor. That is, ITEs are:

• Navigable. The apps provide methods for quickly and intuitively navigating, reviewing, and collating collected thinking.
• Organizable. ITEs provide methods for managing thought structure and metadata, including classifying, tagging, summarizing, and visualizing collections of thoughts.
• Transformative. ITE apps provide methods for defining and refining thoughts, and providing ways of querying, combining, and referencing that thinking.
• Extensible. A rich feature set and/or plugin ecosystem enabling customizability and extensibility for a variety of specific use cases.

ITEs give you as many options as possible for exploring your notes. A common feature is “ quick open.” The quick open workflow is usually something like this: invoke a hotkey (e.g., cmd/ctrl+o), start typing, and watch as the app provides auto-complete suggestions of the note you’re looking for. If none are found, there’s often a way to quickly create the note with the title you’ve written from the same interface.

Some apps have introduced a graph view, visually illustrating the links between notes and allowing you to traverse these links as a way of exploring your collected thoughts.

Others provide date-focused interfaces, allowing you to see the notes you’ve created on a given day or to set due dates or recurring dates for certain thoughts.

Still another crucial feature is the ability to navigate from the content of the notes themselves. In other words, links! ITEs provide ways of opening and previewing the content of items you’ve linked to in the current note.

ITEs provide multiple ways of organizing thinking. Folders (or groups) and tags are the basic features available to most modern operating systems. ITEs typically add saved searches, starring or flagging notes, and custom metadata, allowing you to sort and classify your thinking in nearly any imaginable way.

Another important feature is easily linking between notes or sections of notes. Sometimes this is available via wikilinking, in which a user can create a new file or search for related files simply by invoking a special syntax (typically two square brackets, or [[). Otherwise, linking may be available via app-specific “deep links” between notes.

Through features such as embedded content, embedded searches, and through efficient and intuitive user interfaces for the organizing features I mentioned above, ITEs make it easier to process thinking.

For instance, you might have a tag for drafts. An embedded search for that tag in an “inbox” will give you a dynamic view of all current drafts. Finish with a piece of writing? Remove the tag, and the list automatically updates.

Thought extraction and composition are additional examples of transformation in ITEs. These features let you select a subcomponent of one thought (e.g., a sentence, a quote, or some other snippet of text in a note—these are sometimes called “blocks”) and extract it into its own standalone idea. This lets you add density to the first idea. Similarly, composition features help you piece together multiple thoughts from different sources, combining them into a new whole.

No one uses the same ITE twice, as it’s not the same ITE—and they’re not the same person.

This is the final frontier of ITE design, development, and use. To enable extensibility, ITEs allow users to customize the features of the tool and the ways in which they can structure their thoughts within it.

Ultimately, exactly how an ITE will help you use knowledge depends on the use case. This is where the extensibility and customizability of an ITE becomes particularly important. You can tune an ITE to be exactly what you need it to be for particular kinds of knowledge work. With a few different plugins, templates, and automations, one ITE can be transformed into many different specialized tools. Many users of the apps mentioned earlier have already done this by fashioning the ITE into their very own.⁸

Projects in ITEs will also have domain- and context-specific data, information, knowledge, and wisdom. By letting users customize the structure and metadata of the thoughts they capture, ITEs help people frame their ideas and thinking as data, and give them tools to make sense of that data. This data is specific to the person and their context(s) of use. This is why extensibility is so important: no one uses the same ITE twice, as it’s not the same ITE—and they’re not the same person.⁹

Due to extensibility, it may make more sense to think of tools like Obsidian and Roam as ITE Operating Systems. They provide a foundational layer that, through use, get organically shaped into ITEs purpose-built for each user’s needs.

In fact, we are starting to see examples of these ITE extensions for specific use-cases. These environments are sometimes shared and downloadable. Here are a few examples:

I have designed an “Integrated Qualitative Analysis Environment” as an Obsidian vault. See more here.

Federico Viticci ( @viticci), Editor-in-Chief of MacStories, writes an annual book-length review of the iOS and iPadOS operating systems each summer. His approach for iOS 15 and iPadOS 15 in 2021 used a highly specialized Obsidian vault. He has detailed it extensively on AppStories and Connected episodes (e.g., https://www.relay.fm/connected/344, starting at 1h4m), and via paywalled content on Club MacStories.

Viticci’s put together a system that makes it easy for him to keep his personal notes on the iOS 15 operating systems alongside Apple-provided documentation, screenshots and recordings of new OS features, and so on. His system allows him to organize writing and research tasks inline with the exact pieces of work they relate to. He’s also commissioned specialized plugins that help him collate and publish the thinking he’s working on. The approach seems to have streamlined a variety of aspects of his review process, creating an environment tailored specifically for Viticci’s needs.

Many Obsidian and Roam users have created dedicated environments to study specific texts, such as the Bible. These set-ups usually involve providing a way to navigate to and reference passages easily, while extracting out details and concepts to find common themes and interesting insights. For some examples in Obsidian, see this thread on the official forum.

Mark Robertson ( @calhistorian) has set up an extensive Roam graph to support his teaching methods. He uses tagging to add dates and other metadata to historical phenomena that he then navigates in real-time while presenting lectures to his students. This allows his students to ask questions that Mark can dynamically answer with quick pivots around his historical database.

As I’ve already suggested above, I think we’ll see communities of users craft downloadable ITEs specific to certain kinds of work or workflows. A variety of these are already available for note-taking and project management in Obsidian. We have also seen some starter environments for certain areas of knowledge, such as this one for history students.

These kits are a kind of “hypertemplate.” They provide a kind of pre-fabricated foundation for knowledge work that users can share to help others get started. At the same time, power users will use these to capture many aspects of their work (not only content, but also the environment itself) in template form for automation purposes.

One of DEVONthink’s primary selling points for years has been the tool’s built-in AI. In fact, author Steven Johnson wrote about it in 2005. Essentially, DEVONthink builds an understanding of the documents you store in the app. It can then use that understanding to suggest which documents are similar to one another and to automatically organize new documents into the folders you’ve already created.

Doubtlessly, we’ll see more features like DEVONthink’s AI in this new era of ITEs. An early example is Psionica’s Dual, an in-app chatbot of sorts that uses the thinking you’ve accumulated to respond to queries and questions in natural language.

First, I think we’ll see more ITEs in education. Instructors are already building ITE-based resources and may task students with working from an ITE in coursework in order to teach them the subject matter and knowledge management skills at the same time.

I also think we’ll see more education on ITEs. These are complex tools, full of potential. Good design can make it easier to jump in, but you never know what you don’t know. We’ve already seen a number of thinkers create courses and guides on ITEs and the theory and techniques behind them. I suspect these will continue to proliferate, and will become increasingly infused with “softerware” (discussed further below).

We shape our tools, and they shape us. The emergence of ITEs has necessitated the development of techniques that help leverage the NOTE framework for knowledge management and innovation. These techniques, in turn, are being shaped into tools themselves via ITE extensibility.

For a personal example, I wanted a fast way of collating and reorganizing information I had already collected in order to write something new. So, I developed the idea of a “workbench note.”¹⁰ The idea was to have a temporary staging space for collecting ideas that I could then remix and add to. This note was given a special name. This allowed me to leverage Obsidian’s backlinks and mentions feature. When I read an interesting thought while reviewing my notes, I tagged it with the workbench note’s name, allowing me to collect these ideas as mentions of the transitory workbench note.

This approach worked well enough that I wanted a more robust way of working with it. So, I built the Workbench plugin, which provides commands for copying and linking to items across your ITE in Obsidian. This plugin abstracts the workflow I described above into software.

The actor-network ripple didn’t stop there, however. When @mgmeyers released the Kanban plugin, another user realized that Workbench’s features could be used to quickly populate a Kanban board.

In my experience with ITE communities, other examples of interactions between software and technique abound. I think we’ll continue to see a proliferation of ITE “softerware:” well-documented and replicable techniques for working with ITEs in specific ways. In turn, I think we’ll see developers adapt these techniques into tools, which will lead to new techniques.

Multiplayer Roam. Shared Obsidian vaults. Heck, there’s even a nascent ITE tool using collaboration as a strategic differentiator. Collaboration is coming to ITEs.

There are a few outstanding questions about this, however. As I mentioned earlier, thinking is intensely personal and contextual. This leads to numerous challenges. Collaborators will need to figure out protocols for making changes to their peers’ ideas. Syntaxes will need to be agreed upon to make sure labels, tags, and so on are standardized. Version control will need to be managed such that conflicting changes to a shared ITE are easily resolved. The list goes on. Designers, developers, and facilitators will need to resolve these issues before collaboration can really take off.

In this article, I’ve argued that a series of recently-launched notes apps are actually the next era of Integrated Thinking Environments. To help understand ITEs, I have described a simple framework, NOTE: ITEs make thinking more navigable, organizable, and transformable while being extensible to specific, personal use-cases. I discussed these dimensions of ITEs in some detail, then suggested some of the ways ITEs will conitinue to evolve in the near future.

One quick caveat to my thinking here: I’m mostly familiar with Obsidian and its user communities. There may be dimensions of theory and practice on ITEs found in other tools that I’m missing out on here. If you can think of any, I’d love to hear about it.

So what? Hopefully, the NOTE framework will give you some ways to think about your own approach to… well, thinking.

If you’re not already using an ITE, consider it! It may help you capture and use the knowledge and thinking you’re already doing in your life and work. The NOTE framework gives you some things to consider when choosing between existing ITEs. How will you navigate, organize, transform, and extend them?

If you are using an ITE, NOTE may be used as an evaluation framework. How does your ITE serve you along the four dimensions of navigation, organization, transformation, and extension? Where do you experience friction? What tools or techniques might eliminate that friction to make it easier for you to manage and innovate on your knowledge?

For ITE designers: the NOTE framework may be used as a sort-of rubric for feature design. How does your environment help users navigate, organize, and transform the thinking they put into it? How might they extend it to perform these tasks even better for their context?

The good news for everyone: this new ecosystem is thriving. Programmers can keep their IDEs—I’m happy with my shiny new ITE.