Once you hit a certain age you become permanently unimpressed by a lot of shit.
Unknown
I have written about Switching from DEVONthink to Obsidian and did a posting on Obsidian itself, but that tells you only what the software could do. Let’s look a bit on how to make it useful, given that is provides a high degree of freedom — and that can paralyze. Well, within the bounds of showing a note-taking/knowledge-management app that contains private information.
Overall Approach
What’s it for? For me, an extended brain, esp. long-term storage (sources, information, etc.) that includes details. I have a pretty good memory for «wasn’t there something like …» or «I have done something similar …» but I am not good on details. And in line with David Allen’s «your head is for having ideas — not for holding them» also a place for collecting ideas. Also for managing my work, incl. notes about projects, theses, teaching, etc.
Number of Vaults
In general I think one vault for all information is probably best, given that a lot of the information is usually related. At least with my current job in academia. I am also privately interested in pretty much the same topics and I like to refer to fictional sources to explain things or just provide some anchors to remember the information. Also, when using Obsidian as a logbook, keeping everything in one vault allows me to link to specific entries. E.g., if I write down that I have read a book on a certain day, I can link to the sources file of that book (where I store quotes and notes about the book). Via backlinks I can then always see from the sources file when I did read the book (link points to the day entry).
However, I made two exceptions so far:
- A separate Vault for Blog Postings. Reason being that with postings I write for others (and the information gets deleted after publishing), while the main vault is for myself.
- A separate Vault for Literature Research. I’ll write another posting about it (there are cool things you can do with Scopus csv Export and R to create notes to each relevant hit). Reason here is that sources in the main vault is for things I have actually read. The Literature Research has another goal — get an overview of relevant work and select the crucial ones to read (essentially Zotero on steroids).
I might also (reluctantly) split the main database if the amount of data becomes unwieldy. But I do not expect this to become a problem. Obsidian works with linked data and should scale nicely. Also, the largest files are not in the Obsidian Vault but in external folders. So there is only a simple link to these folders or files in Obsidian itself.
The following comments refer to the main Vault. I have written about the Vault for Blog Postings here: Using Obsidian and R for a Blogging Workflow.
Data Storage
While the notes are all stored in Obsidian (would not make sense otherwise), I differentiate between attachments I absolutely need to have in the Obsidian Vault and those that are better put in external folders. So far, only images I need as part of the notes (= are embedded in a note) are in Obsidian. Almost all other files (PDFs, videos, etc.) are stored outside of Obsidian. Exceptions are a few small video files (also part of notes) and a few temporarily embedded PDF files. All other files I mention in my notes are links to an external folder.
The reason is simple. The main Obsidian vault is 2.45 GB, due to the amount (and size) of images I have used in the notes (2.39 GB are in the Media folder). That sounds much but does not even include the main part of my notes, which I have not included yet. And it is vanishingly small compared to the external folder which contains all the files I have linked to from the vault. That one is 68.04 GB. It contains higher quality versions of my works, reference images, sounds, texts (everything read and included as sources, and a best-of selection of videos).
If I would not store the files in an external folder, my backup (or mobile copy) would be 70+ GB. This way, it is much easier to handle. I can copy the folder to an USB stick to have it available on the go.
BTW, that is not the only external folder. I also have a folder for larger files (mostly videos of courses I have given) which has 28.61 GB. Then there is my Library folder, which is 237.69 GB and contains fiction (80.75) and non-fiction (156.92 GB) literature. All in all too large to have them included in Obsidian — I could not keep the vault on a mobile device.
BTW, linking to the files is fairly easy. At first, I used an R script. I just put the file name in a note in the Obsidian vault, the R script reads the file, then searches for matching filenames in the external folders and creates links to that file. Once it has found all files, it overwrites the markdown file. One quick command (thanks R autocompletion) and I have the links (to one or a hundred files, does not matter much). Then I noticed that dragging a file in a note while pressing OPTION likewise creates a link to that file.
Oh, and you can even embed images via links. Still, for images on notes, I bring them into the vault. The note would likely be useless without that image, so it has to be included.
BTW, the links usually look kinda like this: [File Name](file:///Users/USERNAME/Documents/VAULTFOLDER/SUBFOLDER/SUBFOLDER/FILENAME)
One caveat — with external folders, you have to be consistent with the file and folder names. You cannot simply rename folders, otherwise the links do not work anymore. Obsidian cannot help you as the folders are outside of the vault itself. However, there are ways to quickly change the links when you have changed a folder/file name. I found that using an external editor like BBEdit and using a Multi-File Search works very well. However, you can shred your notes if you are not careful. First perform a multi-file search, look careful at the results — that is what it will replace. Then use replace if and only if you are sure only the correct entries are changed. (Constructed example, you change the path from «stuff» to «work», search-replace for «stuff» would probably change a lot of entries in notes that are not part of links. Including the slashes with «/stuff/» might be a bit safer, but still, look at the hits first.)
This said, all embedded images are in a Media folder (which is also the destination folder for attachments via Settings > Files & Links > Default location for new attachments > In the folder specified below; then set the Attachment folder path). I then manually move it in one of the subfolders (there are folders for, e.g., general images, with subfolder for the year I have added them; my own images, and images from sources, e.g., books I have read). Given that Obsidian can auto-update the images, moving them is not a problem. Moving them via the finder can be, if there are files with the same name (I use Wikilinks — Settings > Files & Links and Automatically update internal links).
Folder Structure
I am a fan of folders, given that I have disparate content in the same Vault. Besides the Media directory for images (and a few videos/sound files), there are folders for:
Database Maintenance
Contains Templates directory, backup of the Obsidian CSS snippet file, notes with tips on using Obsidian and my own solutions on using it, the Markdown files to create documents links (and transform input text into sanitized quoted text, see below), tips on search operators, notes on styling notes entries (incl. how to use Markdown, etc.)
Creative Projects
Contains subfolders of different areas (I am a jack of all trades, ace of one or two), so e.g., Photography, R, Sewing, Tinkering, Writing Fiction, Writing Non-Fiction, etc. Also note files with overviews, incl. those automatically generated by the dataview plugin. The folder structure is helpful here, you can simply create a LIST FROM “My Creative Projects/Photography Projects” and sort the results alphabetically with SORT file.name ASC.
The notes files themselves contain the title, tags (area, status), and the notes themselves (with embedded images that are stored in the media directory of the Vault). Newest entries come first, the longer you scroll, the further you go into the past. Saves scrolling when adding new ideas.
Each area also has an archive containing realized ideas (see below). This way, the Vault not only holds ideas for projects but also documents realized projects.
As not every idea is immediately worthy on an entry, each area also has a Backlog file. It just contains the ideas, each idea has its name as header with the note after it. This way, all smaller ideas are in one file (ideas quickly skimable with the outline view). Also makes adding ideas to the collection easy, just pressing command + o to open the find view, then writing, e.g., «tink back» (autocompletion is a charm) and press enter. The best collection does not work if it sucks to use it. And luckily, Obsidian is really useful here.
Information about myself
A few folders with personal information, e.g., health information, personal notes, clothes sizes, etc. Also contains the logbook files (via Daily notes Core Plugin, you can setup where Obsidian expects the note via New file location, see below).
Work Information
Given that I am more or less a knowledge worker, it makes sense to store information here. Especially considering that my work is also more or less my hobby (despite recent turbulences, Changing Work-Life Balance and Calendar Time Management). It contains a few overview notes (e.g., list of publications, advised theses, etc.) and folders for projects, planned publications, teaching, theses, etc. Given that most of the needed files are not images but PDFs, Keynote files, etc., they are in an external folder and linked in the notes themselves. Otherwise the Vault would be way too large. Even mere slides add up to over a GB for a lecture with 15 presentations.
General Knowledge/Skills Notes
A large part of the Vault is a folder with lots of topic-specific subfolders. A bit inspired by a book of Quotations I did a while ago. I tried to find distinct categories (e.g., Design, Ethics, Psychology, Travel). The note files contain just that — notes on these topics. However, they usually do not contain quotes from texts I have read but links to the sources files, akin to embedding images (see below).
Sources Notes
Another large part — folders for different media types (e.g., articles, fiction books, magazines, non-fiction books, etc.) with notes containing the title, tag (might remove it), link to the PDF file (external folder, another copy is in the library folder to have a backup in case of accidental edit/deletion), APA citation and BibTeX if I ever cite it, a summary (if needed) and quotes.
The individual quotes have a lowest level section header (######), make them easily skimable in the outline, and are written as quotations (using markdowns > ). They also each contain the source (authorname_year and in case of books also the title). BTW, the cssclass YAML header is used to give a source note a different background. Made more sense when I was using a light mode template. Not sure whether I keep it. Simply something like:
.markdown-preview-view.sourcefile {
background-color: #0D171F;
}
.markdown-source-view.sourcefile {
background-color: #0D171F;
}
to your snippets.css file. BTW, there has to be an empty space after the cssclass: in the YAML (cssclass:sourcefile does not work, cssclass: sourcefile does). Sucks if you use the wrong coding style and it takes … a while to identify the problem.
This way, I can easily cite an individual quotation in other notes. Just link to the source note (e.g., ![[wiseman_2004 …]]) and then using the # to write part of the heading or select it via the dialogue. See the image below.
As for writing the quotes in quotation marks, I used to use BBEdit for it. Just copy the highlighted text from the book (incl. digital notes) in a text file, then replace empty lines with the source and an empty line. Then search and replace the line breaks (\r) with line breaks incl. markdown quotation marks (\r> ). Then replace all lines which are empty save for the markdown quotation mark with empty lines (otherwise it is one continuous quotation). Works, but still a lot of effort. And I had to deal with ligatures and the like. Now I use an R script that makes the text citable. It expects the source as the first line, one(!) line break between each quote. It does the rest itself (reads and saves to the Make Citable note which is shown in the bottom right of the Obsidian window, see screenshot above.)
The use of individual note files makes sense for books or articles, but not sure about smaller texts (e.g., from a news report). So far, I have collection notes for specific newspapers (e.g., «Die Zeit»). I have also started to use generic source files for smaller stuff I see on the web (e.g., «2023-02 Web Sources»). Using a file per month should work to keep the amount manageable. BTW, it does not matter when it was published online, only when I encountered it. On the plus side it should allow me to see over time when I was interested in which topic. The idea is to have a way to store information so it can be added quickly, used quickly, and found quickly. Should work.
Top Level Files
There are also a few notes files on the top level of the Vault. These include the start page of the Vault (pinned) which provides quick navigation to the overview files of each folder area and a couple of specific files (the ones I am working on). Also the main creative projects at the moment, links to external folders to which I add images and other files (collection inbox), other vaults (e.g., blog writing, literature research), and external folder links to each folder of my library (e.g., articles, books, reports) and the reference images main folders.
There are also canvas boards to get a high-level view on a few things. E.g., regarding projects (see Prioritizing Projects in Organizing Creativity, p. 57):
Integration with External Folders
I have already written about the reasons for external folders above (tl;dr: Vault becomes unwieldy once its larger than a couple of GB). Given that you can easily link to folders (or specific files), the external folders are linked to from the Vault.
The main externally linked folders are:
External Documents Folder
Contains mostly source or work files. PDFs of books and articles used in sources, keynotes and pdfs of lectures, pdfs of advised theses, etc.
Larger Files Folder
Contains mostly large videos, e.g., my online lectures (thanks, Covid) or other presentations I did that were recorded.
Library Folder
I prefer my reading digitally, much easier to carry (moving sucked until I went digital) and much, much easier to export interesting text. But it adds up if you are a digital squirrel. Sorted into Fiction and Non-Fiction, the later further differentiated by type (e.g., books, articles, reports, etc.).
Given that Finder can look for file names (name:FILENAME), searching for particular books is very fast (almost all are named «authorname_year Title»). If I am looking for a particular quotation, I can go to the subfolder (e.g., «books/a») and do a fulltext search over multiple files (just use the Finder search, entering what you are looking for in quotation marks). Spotlight is surprisingly fast.
Images (as reference)
The images I have used in notes are in the Vault, but there are lots of other images (about 96k, again: digital squirrel). I have started to rename them to describe the content (works if you want to binge watch a couple of series, but not only binge watch them). The subfolders are similar to the General Knowledge/Skills Notes categories (see above), which makes it easy to find the right image. (I can’t remember faces, but I have an excellent memory for movies and images.)
Prepared Daily Notes
Huh, I guess that deserves a separate posting, look for it titled: Obsidian Prepared Daily Notes.
It’s only the beginning …
I have just started to use Obsidian, but like written, I am impressed. Especially the integration with other files/folders is impressive, which also allows for the use of R (or any programming language) to create/edit/move files.
I think there is no perfect note-taking/knowledge management tool — it has to fit to your work/style and where you are at the moment. Also tools usually rise, enjoy some time working very well, until they fold (Circus Ponies Notebook), become distorted (DEVONthink) or untenable subscription models (Notability. Whut? I only rent access to my data?). But at least for the moment (and hopefully for a long time), Obsidian seems to be one hell of an app.
