Examples of using the Scrapbook platform.
Scrapbook overview
Scrapbook is a software platform for managing personal information. Information is anything that's important to you. For us that includes books read, dinners with friends, correspondence, travel summaries, people, and music to name just a few categories. The goal of the Scrapbook platform is to capture, store, and easily recall this information securely, whenever, and wherever.
Why are we writing about Scrapbook (again)?
Scrapbook is a research project. We write to report how we are doing to those of you who might be interested or are pursuing something similar. Above all, we write – even if in the smallest way – to inspire people to take control of their personal information and make it work for them. Scrapbook is about sovereignty over your data. Scrapbook for us is about managing our personal information and making it useful to us, not ceding it to any platform that locks it up to monetize it.
Other posts about Scrapbook:
- 2022 - A Natural Language Query Parser for an Information Management System Called Scrapbook
- 2019 - Digital Scrapbook: A Curious Cross Between a Cabinet of Curiosities and a Vademecum
- 2017 - A Personal Information Management System: Introducing Scrapbook
In some ways Scrapbook is analogous to a physical filing system. In a filing system, you can think of one cabinet as a collection. One drawer in a cabinet is a category. A file folder in a drawer is an item in a category. A file folder may have photos, documents, and other items in it that explain the item.
In the digital world, Scrapbook is structured hierarchically like this:
- The Scrapbook platform contains collections
- Collections are isolated from one another.
- Users may have access to one or more collections.
- Collections contains categories
- Common examples: books, people, recipes, events, etc.
- Categories can be anything you like.
- Categories contain items, and each item is described by fields of information.
- A field is one of the following
- A default field that applies to all categories. For example, "date", "id", "description", or "title".
- A custom field that applies to specific categories. For example, the book category has a "publisher" field.
- Optional types can be used which act like subcategory or synonym of the category.
- Items belong to a category in a collection.
- After an item is created, it can be edited, deleted, moved, or cloned (even between collections a user has access to).
In this post, we explore scenarios illustrated in two of our collections: "MyJournal" and "Lostvibe". MyJournal is our primary collection containing information relevant to our everyday lives. We write to and query MyJournal every day. Lostvibe is music-themed collection focused organized by cover art, among other typical attributes like recording artist and date.
So, you created a digital file system. Anything else special about it?
Five reasons why the Scrapbook platform is unique.
- Accessible anywhere. The collections in Scrapbook are accessible on any device that you can browse on, and on multiple channels (Teams, Skype, Slack) via a chatbot interface as well as Alexa. A physical file cabinet or even a file system is not accessible in this way.
- Searchable. Ever wonder the last time you had dinner with a friend? Need to look up your thoughts on a book you've read five years ago? Or what was the name of the restaurant you ate at during a recent trip? The Scrapbook platform makes these questions easy to answer. It was designed from the start with searchability in mind. Sometimes, a simple text search suffices. Other times, a search might be scoped to a category, over a period of time or place.
- Natural language query. Natural language means asking something of a system in plain English. Once Scrapbook acquired this functionality, it's become our default way of searching. Natural language queries can be written or spoken. Here are some examples: "Show me my last trip in England", "What about in Greece?", or "What about in 2018?". Scrapbook maintains context between natural language queries.
- Create narratives. An alpha-user of Scrapbook said that Scrapbook wove a "red thread" through the entries she'd made about her grandfather connecting them in a way that told a story. We like that idea.
- Items have written descriptions that are authored in markdown and rendered to HTML. But there's more. The Scrapbook platform has "edges" or relationships between items – the red threads.
- Edges have a type and direction. Types are customizable. We use types like "is about", "is related to", "is from", "created by", and "experienced by". The direction of an edge can be from item A to item B, vice versa, or bidirectional.
To illustrate, a trip can be related to the hotel where you stayed, the restaurant where you had lunch, a song you liked, the friends you traveled with, and the party two years before when you met. - In addition, we support hashtags as another level of connecting items.
- Privacy first. We initially built Scrapbook as a closed (non-public) platform initially. We have since added a few basic sharing options, but the thrust of Scrapbook is as your personalized collection – not a social media feed.
How big is it?
We try to live paperless, keeping only essential documents, our passports for example. Any piece of information which can be printed or viewed on demand we digitize. And not only paper, this applies to many kinds of objects – think trophy, perhaps a souvenir, objects which may have been special at one time, but later become a burden to keep around. We scan or take photos and enter the item into Scrapbook. The original object may be given away or recycled and we don't feel too bad about it at all.
Let's talk first about our MyJournal collection, which is our main collection. The 10,000 items in MyJournal is small and large at the same time. It's small in terms of collections we are all familiar with such as Wikipedia or many popular music and film databases. On the other hand, a collection of 10,000 items is large on a personal level, being equivalent to 27 years of diary entries. The collection includes 90 GB of backing data: 44,000 documents, photos, audio, and video assets. Each asset is associated to a MyJournal item. Each of the 10,000 MyJournal items has text that describes it. To be precise, the text – contextual tags, fields, markdown, and relationships – ranges from a couple of hundred bytes to megabytes.
Note: If you were to consider MyJournal as a diary or perhaps a personal encyclopedia, the narrative word count alone stands at over 100,000 words. For comparison, Mark Twain's "The Adventures of Tom Sawyer" is about 71,000 words. So, "The Adventures of Scrapbook" is well on its way.
Our music-themed collection "Lostvibe" contains about 9,000 items. It has about 125,000 backing assets (imagery) totaling 28 GB. The Lostvibe collection has categories including "album", "artist", and "playlist", which link to Spotify.
The graph below shows the growth of items in the two collections.
Graph of number of items managed in two Scrapbook collections.
What's new?
Since our 2017 post and 2019 post, the following are new capabilities of the Scrapbook platform:
- Moved to version 5 of the Scrapbook platform.
- Scrapbook supports multiple collections. For example, we move between multiple collections during our normal workflow, two of which are the MyJournal and the Lostvibe collection discussed in this post.
- Users can be authorized for one or more collections.
- Reworked web experience so transition between desktop and mobile is more seamless.
- Using the same presentation code for both has been challenging and at times, not optimal. We rely on Bootstrap to help us.
- Experimented with more channels for consuming Scrapbook data, including Alexa, Teams, Skype and others.
- Made relationships between items first-class objects.
- An item is represented by an Azure Cosmos DB JSON record. Think of it as a structured blob of text with a unique ID. Relationships between items are similarly represented.
- With edges now defined as records, our queries became easier as we can follow the "graph" between items. We can edit relationships without editing items. And, it positions us in the future to incorporate additional graph database capabilities.
- Natural language query.
- Natural language query means asking a question in plain words as you would another person.
- The natural language query capability was forged in our work to implement a Scrapbook 'Skill' service to provide access to the platform through popular voice and messaging channels. The natural language query generation functionality was abstracted out and made into a web service, which we now use in the core Scrapbook web UX as well.
- Geo location and mapping become first class presentation options.
- Each Scrapbook item has a friendly location field and a geo-location field.
- As with the natural language functionality, our initial steps toward location-based queries and mapping capabilities started in our work developing the Scrapbook skill bot channels. That code was then generalized into a web service, consumable from the Scrapbook web UX or bot channels.
- When querying Scrapbook, we have the option of displaying results in "card", "image", "list", and now "map" views.
Ten scenarios
In each of the following ten user scenarios, we'll be using natural language (NL) queries because that's what we use most of the time. However, there are still dropdown controls for customizing search parameters and filters if needed.
Scenario: music lover
You want to explore your music collection by artist origin (country), genre, album cover description, date, and performer.
Example NL queries:
Quickly recall the details of past walks, hikes and other fitness events and get useful stats. Search by title, date, or location. Example NL queries:
Scenario: correspondence cache
Correspondence scenario: shows results starting with the query "Show me postcards".
Scenario: get help
Scenario: food lover
Scenario: travel- Show artists from United Kingdom
- Show items with Cocteau Twins
- Show artists from America between 1960 and 1980
- Show album covers with keyword baby
- Show albums in genre chamber pop
- Show albums by Jah Wobble
- Show albums release in 2008
- Show albums in the 1980s
- Show albums in the 1980s with genre new wave
- Show albums with track titled hallelujah
Music scenario: shows the result of starting with the query "Show me a map of artists".
Music scenario: shows the result of starting with the query "Show me albums with keyword hands".
Many of our music, especially visual, themed posts in this blog, are generate by search our Lostvibe music collection to help us organize content to create the post contents.
Scenario: fitness tracking
Quickly recall the details of past walks, hikes and other fitness events and get useful stats. Search by title, date, or location. Example NL queries:
- Show me the last hike we did in 2020
- Show me hikes we did in Italy in 2018
- Show all hikes in 2007
- Show me hikes within 50 km of Bergamo
- Show me yoga last month
- Show runs with marathon
Fitness scenario: shows the results starting with the query "Show me hikes in south tyrol in 2020"
Scenario: correspondence cache
Digitize, save, and recall correspondence when you want, including physical letters, articles, clippings, and postcards, as well as already-digital versions of these. Correspondence and other easily digitized material is a great fit for Scrapbook. A category like "correspondence" can have types that can be customized for searching. For examples, correspondence of type "article", "invitation", or "postcard".
Example NL queries:
- Show me articles
- Show me Christmas cards
- Show me invitations
- Show me correspondence from 2015
Correspondence scenario: shows results starting with the query "Show me postcards".
Scenario: get help
Shows the built-in NL help system.
Scenario: food lover
This scenario shows how we use a category "food" (with synonyms mapping to food) to track recipes and meals we want to remember. This serves as a recipe tracker and inspiration for future meals.
Example NL queries:
- Show me desserts
- Show me food with pasta
- Show me pasta with colatura
- Show me waffles
Recipe - food scenario: shows result starting with query "Show me risotto with porcini".
See Orecchiette Pasta Dishes and Recipes for our Favorites: Radish Pesto and Tomato-Broccoli for another example of this scenario.
Scenario: book lover
Scenario: contact organizer
Scenario: wine enthusiast
Scenario: book lover
The scenario is a virtual bookshelf, where you can easily find a book and its details. Books that mean something to you, read or not read. In MyJournal we have a category called "book" with their custom properties for author, publish date, genre, and other customizable fields like status (read, not read).
Example NL queries:
- Show me books read in 2015
- Show me books about science
- Show me books by Dawkins
Book scenario: shows results starting with the query "Show me books by Barry"
Book scenario: shows results starting with the query "Show me books read in 2012"
Scenario: contact organizer
This contact / people organizer is meant to link to relevant events and other people tracked in Scrapbook. Who was at an event? Tracking people this way, you get a sense of who really is important in your life. Who you spend time with. Scrapbook has support for standard uses of @ to refer to people.
Example NL queries:
- @NAME
- Show me dinners with NAME
- Show me travel with NAME
- Show me relations to NAME
- Show me people in 2017
- Show me people in Milan
- Show me the last lunch with NAME
- Show me the earliest lunch with NAME
Contact - people scenario: shows results starting with the query "Show me people".
Scenario: wine enthusiast
In our collection MyJournal, we define a category "Drink" and have subcategories for wine, beer, etc. The category could just as easily be "Wine" or if the whole collection is about wine, maybe "White Wine". It's completely customizable. Tracking wines is not just for wine aficionados (oenophiles). There are stories behind each wine: where the wine came from, what the traditions are behind the wine, the type of grape used, and perhaps more importantly, who you enjoyed the wine with.
Example NL queries:
- Show me wines from Piemonte. Where were they?
- Show me wines from South Tyrol.
- Show me wines we had with @Roberto
- Show me wines with Valdobbiadene
Wine scenario: show results starting with the query "Show wines from Piemonte".
For the travel-inclined, a travel category in Scrapbook let's you explore where you have been - geographical areas, countries, regions, cities. It's also useful for travel planning, all the work that goes into planning a trip that you may or may not take. (There could be a separate category called "Travel Planning".) Trips (entries under "Travel") can be connected (i.e., have edges) to cities, people, and events. This is one area where we share items for consumption outside of Scrapbook, especially trip planning when friends ask. Often, we give a trip a hashtag (e.g., #England2016Trip) so we can search by hashtag to group all items together.
Example NL queries:
- Show me Milan in 2019
- Show me Seattle in July 2009
- Show me travel in England
- Show me lodging in London
- Show me restaurants in Bergamo
- #England2016Trip
Travel scenario: show results starting with query "Show me stuff in Milan".
Scenario: museums and exhibitions
We are museum junkies. It bothers us to go to a exhibition or museum and walk out without remembering anything. Blank slate! Scrapbook was developed around the concept of recording something learned from events experienced like going to a museum. After a couple dozen entries, this category starts to become interesting because while in a museum viewing a work of art, we'll ask ourselves if we've seen it before and we can ask Scrapbook. In this case, Scrapbook helps us appreciate what we are seeing.
For museums and exhibits, we include any rich data, brochures, and documents. You would be surprised how unreliable (after say a few years), museums are for tracking past exhibitions.
Example NL queries:
- Show me museums in Bergamo
- Show me exhibitions in Vienna
- Show me museums in Hartford Connecticut
Museum - exhibition scenario: show results starting with the query "Show me exhibits".
