Annotations of The notion of data and its quality dimensions - Fox, Levitin, Redman - 1994 from 20240815 at 11:17

The rapid proliferation of computer-based information systems has introduced an army of new and unsophisticated users to computers. Because they are often less well- trained, such users tend to feed systems erroneous data. Furthermore, inexperienced users are less able to recognize and deal with erroneous data than experienced users, and there- fore can be victimized by it. These problems drive system and process data quality improve- ments to reduce error rates in data input, data output, and data processing, and to make processes more tolerant of data errors. (p. 2)

The authors here imply that there are dimensions of quality beyond data values, models, or representation: i.e., dataset qualities. Clearly this kind quality — the authors offer “usefulness”, though they don’t define what that means — is an emergent, systemic property of data model, values, and representation qualities.

Annotations of Data quality in information systems - Brodie - 1980 from 20240815 at 11:00

Data quality is a measure of the extent to which a database accurately represents the essential properties of the intended application. Data quality has three distinct components: data reliability, logical (or semantic) integrity, and physical integrity . (Data quality in information systems - Brodie - 1980, p. 3)

Annotations of Where Good Ideas Come From- The Natural History of Innovation - Johnson - 2011 from 20240814 at 15:48

Johnson describes how Darwin’s fundamental theory of evolution did not appear all at once, but instead shows up in Darwin’s personal notebooks in different ways for years before he finally recognizes and crystallizes his thinking into a full theory:

It is simply hard to pinpoint exactly when Darwin had the idea, because the idea didn’t arrive in a flash; it drifted into his consciousness over time, in waves (Johnson, 2011, p. 81)

I recall experiencing similar “waves” as I developed a few of my successful contributions to scholarship. Ideas show up over and over again. There is no one point where an idea forms, except in the retrospective we tell ourselves about the idea.

This suggests the importance of the recursive systemic view. Ideas really are made up of smaller ideas, and those made up of smaller ideas. Some ideas come directly from other thinkers, while others come from observation of the world, while others come from creative abduction, and still others come from editing and feedback. These components all interact in a system and any one small change produces perturbations in the form — but if an idea is powerful enough, it will emerge, one way or another (“good moves in a design space”).

Ideas are powerful enough when they fit a niche in the broader idea ecosystem — that is, they can draw on the right untapped or uncompeted-for resources, and they solve the right problems.

We can track the evolution of Darwin’s ideas with such precision because he adhered to a rigorous practice of maintaining notebooks where he quoted other sources, improvised new ideas, interrogated and dismissed false leads, drew diagrams, and generally let his mind roam on the page. We can see Darwin’s ideas evolve because on some basic level the notebook platform creates a cultivating space for his hunches; it is not that the notebook is a mere transcription of the ideas, which are happening offstage somewhere in Darwin’s mind. Darwin was constantly rereading his notes, discovering new implica tions. His ideas emerge as a kind of duet between the present-tense thinking brain and all those past observations recorded on paper. Somewhere in the middle of the Indian Ocean, a train of association compels him to revisit his notes on the fauna of the Galápagos archi pelago from five months before. As he reads through his observations, a new thought begins to take shape in his mind, which provokes a whole new set of notes that will only make complete sense to Darwin two years later, after the Malthus episode. (Johnson, 2011, p. 83)

This is a magnificent demonstration of a powerful knowledge innovation system and practice.

Annotations of 𖠫 Where Good Ideas Come From- The Natural History of Innovation - Johnson - 2011 from 20240812 at 10:21

Page 17

both the city and the Web possess an undeniable track record at generating innovation.2 In the same way, the “myriad tiny architects” of Darwin’s coral reef create an environment where biological innovation can flourish. If we want to understand where good ideas come from, we have to put them in context. Darwin’s world-changing idea unfolded inside his brain, but think of all the environments and tools he needed to piece it together: a ship, an archipelago, a notebook, a library, a coral reef. Our thought shapes the spaces we inhabit, and our spaces return the favor. The argu ment of this book is that a series of shared properties and patterns recur again and again in unusually fertile environments. I have distilled them down into seven patterns, each one occupying a sep arate chapter. The more we embrace these patterns—in our private work habits and hobbies, in our office environments, in the design of new software tools—the better we will be at tapping our extraor dinary capacity for innovative thinking.

Does the city and the reef follow TAP? If so, does the same pattern apply to e.g. conduits collected by Darwin? What does TAP and Johnson’s book agree upon?

Page 18

. In the lan guage of complexity theory, these patterns of innovation and cre ativity are fractal: they reappear in recognizable form as you zoom in and out, from molecule to neuron to pixel to sidewalk. Whether you’re looking at the original innovations of carbon-based life, or the explosion of new software tools on the Web, the same shapes keep turning up. When life gets creative, it has a tendency to grav itate toward certain recurring patterns, whether those patterns are emergent and self-organizing, or whether they are deliberately crafted by human agents.

Constructal flow must apply here as well.

Page 20

When we look at the history of innova tion from the vantage point of the long zoom, what we find is that unusually generative environments display similar patterns of cre ativity at multiple scales simultaneously.

So is there a “long zoom” view of data design?

Page 102

A recent experiment led by the German neuroscientist Ullrich Wagner demonstrates the potential for dream states to trigger new conceptual insights. In Wagner’s experiment, test subjects were as signed a tedious mathematical task that involved the repetitive trans formation of eight digits into a different number. With practice, the test subjects grew steadily more efficient at completing the task. But Wagner’s puzzle had a hidden pattern to it, a rule that governed the numerical transformations. Once discovered, the pattern allowed the subjects to complete the test much faster, not unlike the surge of ac tivity one gets at the end of a jigsaw puzzle when all the pieces sud denly fall into place. Wagner found that after an initial exposure to the numerical test, “sleeping on the problem” more than doubled the test subjects’ ability to discover the hidden rule. The mental recom binations of sleep helped them explore the full range of solutions to the puzzle, detecting patterns that they had failed to perceive in their initial training period.

Page 105

Thatcher and other researchers believe that the electric noise of the chaos mode allows the brain to experiment with new links between neurons that would otherwise fail to con nect in more orderly settings. The phase-lock mode (the theory goes) is where the brain executes an established plan or habit. The chaos mode is where the brain assimilates new information, ex plores strategies for responding to a changed situation. In this sense, the chaos mode is a kind of background dreaming: a wash of noise that makes new connections possible. Even in our waking hours, it turns out, our brains gravitate toward the noise and chaos of dreams, 55 milliseconds at a time.

Is the notion of phase locking aligned with the two systems of thinking fast and slow?

Page 110

The shower or stroll removes you from the task-based focus of modern life—paying bills, answering e-mail, helping kids with homework—and deposits you in a more associative state.

One strategy for encouraging serendipitous ideas is to switch thinking modes. Again, constructal flow’s diffusion and infusion.

Page 112

While the creative walk can produce new serendipitous com binations of existing ideas in our heads, we can also cultivate ser endipity in the way that we absorb new ideas from the outside world. Reading remains an unsurpassed vehicle for the transmis sion of interesting new ideas and perspectives. But those of us who aren’t scholars or involved in the publishing business are only able to block out time to read around the edges of our work schedule: listening to an audio book during the morning commute, or taking in a chapter after the kids are down. The problem with assimilating new ideas at the fringes of your daily routine is that the potential combinations are limited by the reach of your memory. If it takes you two weeks to finish a book, by the time you get to the next book, you’ve forgotten much of what was so interesting or provocative about the original one. You can immerse yourself in a single au thor’s perspective, but then it’s harder to create serendipitous colli sions between the ideas of multiple authors.

Here Johnson characterizes one of the key constraints on knowledge management and innovation in knowledge work.

Page 118

So far in the chapter, Johnson has discussed the necessity for disorder in the process of coming up with new, unexpected ideas that connect several seemingly unrelated concepts into a solution for a problem. He discusses a couple of the contributing factors for this process: (1) toggling between focused and unfocused states (e.g., by taking a break from a problem you’ve been working on), (2) collecting many ideas and keeping them available to one another (e.g., by storing them in PKM systems or by engaging in intense periods of conceptual diversity, such as a reading vacation [conferences are arguably a source of this]). At this point in the chapter he is shifting to questioning the role of the Internet in these factors of serendipity, reflecting on how the Web affects analog patterns of accidental discovery.

Annotations of 𖠫 Where Good Ideas Come From- The Natural History of Innovation - Johnson - 2011 from 20240812 at 12:10

One way to do this is to create an open database of hunches, the Web 2.0 version of the traditional suggestion box. A public hunch database makes every passing idea visible to everyone else in the organization, not just management. Other employees can comment or expand on those ideas, connecting them with their own hunches about new products or priorities

Crowdsolving platforms are a modern version of this, but I’m not sure it’s proving the utility of the concept.

Page 128

Johnson concludes with two vaguely-described mechanisms for organizational serendipity: (1) a semantic similarity index of all the work everyone is doing, so that efforts on one project might show up to another tangentially-related but organizationally-distinct project; and (2) a crowdsolving platform for ideas. These are regrettably kind of weak. We now know much more about serendipity and its enablers and inhibitors, and they go far beyond this kind of shallow platform-based solution: instead they address culture and power.

Page 221

for the sake of clarity, let’s not blur the line between “individual” and “network” by admitting to the discussion the prior innovations that inspired or supported the new generation of ideas. Yes, it is important that Gutenberg borrowed the screw-press technology from the winemakers, but one cannot say that the print ing press was a collective innovation the way, for example, the In ternet clearly was. So Gutenberg and Berners-Lee get classified on the individual side of the spectrum.

I don’t think I agree with this scoping of the data. It’s an asystemic approach — it requires a belief that later innovations can be separated from earlier ones, and it (perhaps more problematically) hero-worships the supposed inventors. This does not necessarily agree with e.g., ontological design, in which “good moves in design space” will likely be filled by some actor because of the needs and niches of the ecosystem.

Page 231

Why have so many good ideas flourished in the fourth quad rant, despite the lack of economic incentives?

Another important piece missing from this analysis is the choice of “most significant inventions.” This relates to my earlier criticism: the this view of the primacy and separability of a given invention erases all of the little requisite innovations that were necessary to unlock one of these “major” innovations to be possible. This view is a bit reductive, then, because it fails to account for the feedback loops between innovations big and small.

Page 232

That deliberate inefficiency doesn’t exist in the fourth quad rant. No, these non-market, decentralized environments do not have immense paydays to motivate their participants. But their openness creates other, powerful opportunities for good ideas to flourish.

It seems strange to conduct this analysis without considering the other motivating incentives that exist for these innovators/innovations. Many of the innovations listed are the work of scholars — and sure, maybe they weren’t trying to invent some doohickey for a patent that will give them a lifetime of royalties, but they needed to publish good ideas to be considered a prestigious scholar. It would be interesting to take this same analytical approach but consider this and other kinds of incentives as well.

Page 241

If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it. Its peculiar character, too, is that no one possesses the less, because every other possesses the whole of it. He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me. That ideas should freely spread from one to another over the globe, for the moral and mutual instruc tion of man, and improvement of his condition, seems to have been peculiarly and benevolently designed by nature, when she made them, like fire, expansible over all space, without lessen ing their density in any point, and like the air in which we breathe, move, and have our physical being, incapable of con finement or exclusive appropriation. Inventions then cannot, in nature, be a subject of property.

Page 246

The patterns are simple, but followed together, they make for a whole that is wiser than the sum of its parts. Go for a walk; cultivate hunches; write everything down, but keep your folders messy; embrace serendipity; make generative mistakes; take on multiple hobbies; frequent coffeehouses and other liquid networks; follow the links; let others build on your ideas; borrow, recycle, re invent. Build a tangled bank.

It is remarkably telling that in this sentence — Johnson’s concluding call-to-action of this book — his advice for facilitating serendipity is simply to embrace it.

It’s clear that we don’t know how to do this, yet. Not really.

Annotations of 𖠫 Where Good Ideas Come From- The Natural History of Innovation - Johnson - 2011 from 20240812 at 10:58

The secret to organizational inspiration is to build information networks that allow hunches to persist and disperse and recombine. Instead of cloistering your hunches in brainstorm sessions or R&D labs, create an environment where brainstorming is something that is constantly running in the background, throughout the organiza tion, a collective version of the 20-percent-time concept that proved so successful for Google and 3M.

A decent premise for a design principle.

Page 127

One way to do this is to create an open database of hunches, the Web 2.0 version of the traditional suggestion box. A public hunch database makes every passing idea visible to everyone else in the organization, not just management. Other employees can comment or expand on those ideas, connecting them with their own hunches about new products or priorities

Crowdsolving platforms are a modern version of this, but I’m not sure they’re proving the utility of the concept.

Page 128

Johnson concludes with two vaguely-described mechanisms for organizational serendipity: (1) a semantic similarity index of all the work everyone is doing, so that efforts on one project might show up to another tangentially-related but organizationally-distinct project; and (2) a crowdsolving platform for ideas. These are regrettably kind of weak. We now know much more about serendipity and its enablers and inhibitors, and they go far beyond this kind of shallow platform-based solution: instead they address culture and power.

Annotations of 𖠫 Where Good Ideas Come From- The Natural History of Innovation - Johnson - 2011 from 20240812 at 10:45

Filters reduce serendipity

This is too strong a claim. Certainly filters have some effect on the kinds of concepts, someone can encounter, but that may simply mean that they are more available to other clouds of concepts. I don’t think anyone can say that there is a direct relationship between filtering and serendipity.

Page 121

it’s true that by the time you’ve entered something into the Google search box, you’re already invested in the topic. (This is why Web pioneer John Battelle calls it the “data base of intentions.”)

Database queries, as declarations of intention, is a really interesting framing.

Page 123

Google and Wikipedia give those passing hints something to attach to, a kind of information anchor that lets you settle down around a topic and explore the surrounding area. They turn hints and happy accidents into information. If the commonplace book tradition tells us that the best way to nurture hunches is to write everything down, the serendipity engine of the Web suggests a parallel directive: look everything up.

Johnson argues that serendipity systems require that the serendipity-haver “look everything up”; chase every interest to fill out more connections.

The Serendipity of Streams

A neat article about the structure of (digital) streams of information and their propensity for serendipity and innovation.

A stream is simply a life context formed by all the information flowing towards you via a set of trusted connections — to free people, ideas and resources — from multiple networks.

What makes streams ideal contexts for open-ended innovation through tinkering is that they constantly present unrelated people, ideas and resources in unexpected juxtapositions. This happens because streams emerge as the intersection of multiple networks.

This means each new piece of information in a stream is viewed against a backdrop of overlapping, non-exclusive contexts, and a plurality of unrelated goals. At the same time, your own actions are being viewed by others in multiple unrelated ways.

As a result of such unexpected juxtapositions, you might “solve” problems you didn’t realize existed and do things that nobody realized were worth doing. For example, seeing a particular college friend and a particular coworker in the same stream might suggest a possibility for a high-value introduction: a small act of social bricolage. Because you are seen by many others from different perspectives, you might find people solving problems for you without any effort on your part. A common experience on Twitter, for example, is a Twitter-only friend tweeting an obscure but important news item, which you might otherwise have missed, just for your benefit.

[In a stream, t]he most interesting place to be is usually the very edge, rather than the innermost sanctums.

Not sure I agree with this. The author is binding a bunch of factors into “interesting,” but the truth is that there are different kinds of power here, and whether you want to be in the center or at the edge depends on what you’re trying to do.

If you were an amoeba who could consider individuality only in connection with single cells and if you were to ask a sperm whale, made up of thirty quadrillion cells, whether it was one or many, how could the sperm whale answer in a way that would be comprehensible to the amoeba?

𖠫 The Last Answer - Isaac Asimov