↓Skip to main content

Writing is never done

·6 mins
Journal Thoughts Code

I’ve been journaling and scribbling down ideas since (at least) 2010. In that time I’ve used lots of different technologies to capture these thoughts with the least amount of friction I can handle.

The first, was WordPress back when I thought I was going to be a professional blogger. (Oh, we were all gonna be famous.)

Once that Idea died, I moved to DayOne on my iPhone. I was great until it wasn’t.

Then, I used nvalt on my Mac. I loved how quick and frictionless the process of writing became. As long as I was sitting at my computer, that is. It became inconvenient because, let’s be honest, when an idea strikes, my phone is usually the only thing I have close at hand. Nvalt never did have a good “little sister,” in the mobile arena.

Recently, I found GitJournal on my Android. Okay, it has its issues, but come on! It gets the basics right! Bravo! Markdown, Git, no lock-in, metadata in frontmatter.

I can jot down thoughts, and use those notes to update a site built with Hugo.

Now, finally, I think I’ve found the best method for me to both write and make that writing accessible to my family (which is all I care about with these stupid notes. If something should happen to me, I’d like my family to know some of these things about me.)

However, to make all of these old notes (about 3500) play nicely with Hugo, I’ve had to write quite a few one-off bash scripts to clean things up and add the proper frontmatter.

For instance, DayOne saved its metadata at the bottom of exported notes like so:

...

==================================================================================================
METADATA =========================================================================================
creation date: 2014-09-16T16:22:12Z
tags: ['Sketch']
location: {'region': {'center': {'longitude': -84.10077667236328, 'latitude': 39.70843887329102}, 'radius': 75}, 'localityName': 'Beavercreek', 'country': 'United States', 'longitude': -84.10078, 'administrativeArea': 'OH', 'placeName': '4391 Longmeadow Ln', 'latitude': 39.70844}
weather: None

Hugo, on the other hand, expects the metadata to be saved as frontmatter at the beginning of the file like so:

---
date:   2014-09-16T16:22:12Z
feature: /journal/5cb486b62b05fbe6fb772437e7ea8284.jpeg
location:   {'region': {'center': {'longitude': -84.10077667236328, 'latitude': 39.70843887329102}, 'radius': 75}, 'localityName': 'Beavercreek', 'country': 'United States', 'longitude': -84.10078, 'administrativeArea': 'OH', 'placeName': '4391 Longmeadow Ln', 'latitude': 39.70844}
tags:   ['Sketch']
title:  images DayOne Header Images 5cb486b62b05fbe6fb772437e7ea8284 jpeg
weather:   None
---
...

Pretty easy to do, as long as you don’t have 3500 notes to go through.

So quick BASH script seemed to be a good answer

#!/bin/bash
#last update Jan. 15, 2019
DEBUG=YES
#################################################

# Define functions
#################################################
function pause(){
   read -p "$*"
}

function help(){
  echo
  echo "Usage"
  echo
  echo "    md-clean [-h|--help] \\"
  echo "               (-d <directory>|--dir <directory>) \\"
  echo "               [OPTION]..."
  echo
  echo "Options"
  echo
  echo "    -h, --help"
  echo "        Show this information"
  echo "    -d, --dir"
  echo "        optional. \\"
  echo "        define the path of the directory to list. This defaults to the current working directory. \\"
  echo "        (ie. -d=/home/user/directory)"
  echo
}
#################################################

# Parsing arguments passed to the script
# any arguments not defined below will be passed as arguments/options
#################################################
PASSARGS=()
for i in "$@"
do
case $i in
    -h|--help)
    help
    exit 0
    ;;
    -d=*|--dir=*)
    DIRPASSED=YES
    DIR="${i#*=}"
    shift # past argument=value
    ;;
    *)
    PASSARGS+=(${i}) # unknown option
    ;;
esac
done
#################################################
now=$(date '+%Y_%m_%d_%H_%M_%S')

if [ ! $DIRPASSED ]
then
  echo "working in directory ${PWD}"
  dir="${PWD}"
else
  if [ -d "${DIR}" ]
  then
    echo "working in directory ${DIR}"
    dir="${DIR}"
  else
    echo "supplied directory: ${DIR}, does not exist"
    exit 1
  fi
fi

for f in "$dir/"*".md"
do
  declare -gA FM_ARRAY
  FM_ARRAY=()
  declare -gA META_ARRAY
  META_ARRAY=()
  declare -gA MERGED_ARRAY
  MERGED_ARRAY=()
  if [ $DEBUG ]; then echo "filepath: $f"; fi
  FILE="${f##*/}"
  FILENAME="${FILE%.*}"
  EXT="${f##*.}"
  if [ $DEBUG ]; then echo "file: $FILE"; fi
  if [ $DEBUG ]; then echo "filename: $FILENAME"; fi
  if [ $DEBUG ]; then echo "extension: $EXT"; fi
  FRONTMATTER=$(sed -n '/^---/p;q' "$f")
  if [ $FRONTMATTER ]; then
    if [ $DEBUG ]; then echo "file: $FILENAME already contains frontmatter"; fi
    FRONTMATTER_END=$(($(awk '/---/ && ++n==2 {print NR;exit}' "$f")))
    if [ $DEBUG ]; then echo "the frontmatter ends on line: $FRONTMATTER_END"; fi
    START=$FRONTMATTER_END
    cnt=0
    while read LINE
    do
    if [ "$cnt" -lt $FRONTMATTER_END ];
    then
      if [[ "$LINE" =~ .*":".* ]];  then
        LENGTH=$(($(echo $LINE | grep -o ":" | wc -l) +1))
        KEY=$(cut -d ':' -f 1 <<< "$LINE")
        VALUE=$(cut -d ':' -f 2-$LENGTH <<< "$LINE")
        FM_ARRAY["$KEY"]="$VALUE"
      fi
    fi
    cnt=$((cnt+1))
    done < "$f"
    TITLE=$(awk "/./ { if (NR>"$FRONTMATTER_END") {print;exit}}" "$f")
  if [ $DEBUG ]; then echo "the title is: $TITLE"; fi
    SAFETITLE=$(./safetitle $TITLE)
    if [ $DEBUG ]; then echo "the safetitle is: $SAFETITLE"; fi 
  else
    if [ $DEBUG ]; then echo "file: $FILENAME does not contain frontmatter"; fi
    START=1
    TITLE=$(awk '/./ {print;exit}' "$f")
    if [ $DEBUG ]; then echo "the title is: $TITLE"; fi
    SAFETITLE=$(./safetitle $TITLE)
    if [ $DEBUG ]; then echo "the safetitle is: $SAFETITLE"; fi
  fi
  [[ ${FM_ARRAY["title"]+1} ]] && echo "key: title already exists" || FM_ARRAY["title"]="$SAFETITLE"
  for key in "${!FM_ARRAY[@]}"; do echo "$key => ${FM_ARRAY[$key]}"; done
  METADATA=$(($(awk '/METADATA =========================================================================================/ {print NR;exit}' "$f") - 2))
  if [ $DEBUG ]; then echo "metadata startline is: $METADATA"; fi
  if [ "$METADATA" -gt 0 ]; then 
    if [ $DEBUG ]; then echo "file: $FILENAME contains metadata"; fi
    END=$METADATA
    cnt=0
    while read MLINE
    do
    if [ "$cnt" -gt $METADATA ];
    then
      if [[ "$MLINE" =~ .*":".* ]];  then
        LENGTH=$(($(echo $MLINE | grep -o ":" | wc -l) +1))
        MKEY=$(cut -d ':' -f 1 <<< "$MLINE")
        MVALUE=$(cut -d ':' -f 2-$LENGTH <<< "$MLINE")
        if [ "$MKEY" == "creation date" ]; then MKEY="date"; fi
        META_ARRAY["$MKEY"]="$MVALUE"
      fi
    fi
    cnt=$((cnt+1))
    done < "$f"
    # for key in "${!META_ARRAY[@]}"; do echo "$key => ${META_ARRAY[$key]}"; done
    for key in "${!FM_ARRAY[@]}";
    do
      value="${FM_ARRAY["$key"]}"
      MERGED_ARRAY["$key"]="$value"
    done
    for key in "${!META_ARRAY[@]}";
    do
      value="${META_ARRAY["$key"]}"
      [[ ${MERGED_ARRAY["$key"]+1} ]] && echo "key: $key already exists in merged array" || MERGED_ARRAY["$key"]="$value"
      MERGED_ARRAY["$key"]="$value"
    done
    echo "merged array:"
    for key in "${!MERGED_ARRAY[@]}"; do echo "$key => ${MERGED_ARRAY[$key]}"; done
  else
    if [ $DEBUG ]; then echo "file: $FILENAME does not contain metadata"; fi
    END=$(wc -l < "$f")
    for key in "${!FM_ARRAY[@]}"; do
      MERGED_ARRAY["$key"]=${FM_ARRAY["$key"]}
    done
  fi
  if [ $DEBUG ]; then echo "end: $END"; fi
  mkdir -p processed
  echo "---" > "$dir/processed/$FILE"
  for key in "${!MERGED_ARRAY[@]}"; do echo "$key: ${MERGED_ARRAY[$key]}" ; done | sort >> "$dir/processed/$FILE"
  echo "---" >> "$dir/processed/$FILE"
  awk "NR=="$(($START+1))",NR=="$(($END)) "$f" >> "$dir/processed/$FILE"
  tmpfile=$(mktemp)
  cp "$dir/processed/$FILE" "$tmpfile" &&
    awk "NR>"${#MERGED_ARRAY[@]}+2" && /---/ {gsub(\"-\", \"*\")} 1" "$tmpfile" >"$dir/processed/$FILE"
  rm "$tmpfile"
  sed -i 's/\t/    /g' "$dir/processed/$FILE"
  sed -i '$ s/```//g' "$dir/processed/$FILE"
  sed -i 's/tags:[ ]*\["[ ]*"\]//g' "$dir/processed/$FILE"
  sed -i 's/tags:[ ]*None//g' "$dir/processed/$FILE"
  echo "======================================================================"
done

This used a separate script called safetitle to manage stripping non-alpha characters from the title

#!/usr/bin/env bash
#last updated July 23, 2021

# Set some base variables for the script
#################################################
# set this scripts parent path so that all relative paths will be correct
parent_path="$PWD"
input="$@"
shopt -s extglob
input="${input//[^[:alnum:]]/_}"
input="${input##*(_)}"
input="${input%%*(_)}"
# input=$(echo ${input} | sed "s/_\{2,\}/_/g") # use this if you want to keep _'s but don't want more than 1 at a time
input=$(echo ${input} | sed "s/_/ /g")
input="${input:0:99}"
input="${input%%*(_)}"
echo ${input}

exit 0

In this process, I read quite a few of my old poems and entries. Upon reading them, I immediately started thinking of improvements I could make to them.

Poems I thought I had “finished” in 2013 I found inspiration in.

Writing is never “done.”1

  1. My son, beware of anything beyond these. Of making many books there is no end, and much study is a weariness of the flesh. (Ecclesiastes 12:12 ESV) ↩︎

↑