Awesome software, awesome software tricks, and awesome software tools.

BASH

# can be text or integers
for VARIABLE in one two three four five
do
	echo $VARIABLE
done

# Loop over integers
for (( i = 0; i <= 5; i++ )); do
  echo $i
done

# list directory contents and loop over files
for files in $(ls);
do
  echo $files;
done;

sed -- stream editor

The sed utility reads ⃨ the standard input if no files are specified, modifying the input as specified by a list of commands.

  • You can use an online sed browser, https://sed.js.org, the GNU sed live editor to test commands.

You can use sed to parse a semantic version and split out its MAJOR.MINOR version information.

# say you have a tag in Git with the following format
TAG_VERSION="ios-v5.9.0"

# You can strip out the "ios-v" part with this command
SEM_VERSION=$( echo ${TAG_VERSION}         | sed 's/^ios-v//'    )

# Then you can save off the `MAJOR.MINOR` part
MAJOR_MINOR_VERSION=$( echo ${SEM_VERSION} | sed 's/\.[^./]*$//' )

echo "tag $TAG_VERSION, SemVer $SEM_VERSION, MAJOR.MINOR $MAJOR_MINOR_VERSION"
# Result
#   tag ios-v5.9.0, SemVer 5.9.0, MAJOR.MINOR 5.9

tr — translate characters

Convert Dos TO Unix Using tr Command

# Delete characters from the input.
tr -d '\r' < input.file > output.file

~/.zsrc

Use ~/.zsrc to give a friendlier command prompt for use with Git. References:

  • https://git-scm.com/book/en/v2/Appendix-A%3A-Git-in-Other-Environments-Git-in-Zsh
  • https://support.apple.com/en-us/HT208050
  • http://www.masterzen.fr/2009/04/19/in-love-with-zsh-part-one/
  • https://vincent.bernat.ch/en/blog/2019-zsh-async-vcs-info
  • https://arjanvandergaag.nl/blog/customize-zsh-prompt-with-vcs-info.html
  • https://www.eseth.org/2010/git-in-zsh.html
   ⌄ REMOTE           BRANCH ⌄        ⌄ CURRENT FOLDER                      ⌄ FULL PATH                     
## master...origin/master | (master)😎roblabs.github.io #             😎 ~/github/roblabs.github.io

Add this to ~/.zsrc

autoload -Uz colors vcs_info
colors
precmd_vcs_info() { vcs_info }
precmd_functions+=( precmd_vcs_info )

REMOTE=$(git status -sb | grep \#)

# PROMPT gives you stuff on the left
setopt prompt_subst
PROMPT="${REMOTE} |\$vcs_info_msg_0_ # "
RPROMPT=$'$(set_prompt) %(?.%F{green}😎.%F{red}?%?)%f %d '

# zstyle - is the stuff on the right
# zstyle ':vcs_info:git:*' formats '%b'

zstyle ':vcs_info:git:*' formats '%F{240} (%b)😎%r'
zstyle ':vcs_info:*' check-for-changes true
zstyle ':vcs_info:*' enable git

~/.bash_profile or ~/.zprofile

Notes on awesome stuff to put in your Bash profile.

# the very first bash command I ever learned
alias ll="ls -l"

# redirect output of `ls` to CSV
alias lscsv="ls -hgo * | sed 's/[ \t]/,/g' > ls.csv"

# Display active git branch in bash prompt
# http://gregk.me/2011/display-active-git-branch-in-bash-prompt/
PS1="\u@\h:\w\$(git branch 2>/dev/null | grep -e '\* ' | sed 's/^..\(.*\)/ (\1)/') \$ "

HTML

<details>: The Details disclosure element

<details style="border: 1px solid gray;  border-radius: 5px; background: #EEE; margin: 1em 0;" >
  <summary>Example Map 🗺 </summary>
    Insert map here
</details>
Example Map 🗺

Insert map here

<a>: The Anchor element

  • For Use in Markdown — <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a">Mozilla MDN</a>
  • with ' For use in JSON or Markdown — <a target='_blank' href='https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a'>Mozilla MDN</a>

Colors

Colors in Markdown. Useful you want to give a visual description to an asset in your map, or to exhibit a hex code color pallette.

Colors from HTML

These samples in blue make use of a clever idea that colors the Unicode Geometric Shapes such as : ◔, ▰, ▲, ◆, ●, ■. You can also change the size by using adjusting the The HTML Section Heading elements: <h1>–<h6>.

Super simple: Adjust size, marker, and color with a simple bit of HTML.

Color text

  • You can color some some blue text within Markdown.
    • <span style="color:blue">some *blue* text</span>
Solid color fill of Unicode Characters for use in Markdown
  • <h1> <div style="color:red">&#9684;</div> </h1>
  • <h2> <div style="color:green">&#9648;</div> </h2>
  • <h3> <div style="color:blue">&#9650;</div> </h3>
  • <h4> <div style="color:cyan">&#9670;</div> </h4>
  • <h5> <div style="color:magenta">&#9679;</div> </h5>
  • <h6> <div style="color:yellow">&#9632;</div> </h6>

Solid color fill of Unicode Characters for use in Markdown

    • <h1> <div style="color:red">&#9684;</div> </h1>
    • <h2> <div style="color:green">&#9648;</div> </h2>
    • <h3> <div style="color:blue">&#9650;</div> </h3>
    • <h4> <div style="color:cyan">&#9670;</div> </h4>
    • <h5> <div style="color:magenta">&#9679;</div> </h5>
    • <h6> <div style="color:yellow">&#9632;</div> </h6>
  • #f15625, RGB = (241, 86, 37)

<h4 style="font-family:Courier;color: #ffffff; background-color:#f15625;">#f15625, RGB = (241,  86, 37)</h4>

Errata: These seem to work in Markdown in the editor atom.io, but do not render properly in GitHub Pages

Colors from Images

If your Markdown engine fails to render the colored style, then you are still in luck as there are web services that serve up colors as images.

Vector Images from a URL

shields.io offers badges that are

Pixel-perfect, Retina-ready, Fast, Consistent, Hackable, No tracking

  • https://img.shields.io/badge/-0xff69b4-ff69b4 - Set a -label-hexcode and shields.io will generate an SVG as a URL: https://img.shields.io/badge/-0xff69b4-ff69b4
  • 0xf15625 or RGB = 0xf15625. URL: https://img.shields.io/badge/-(241,86,37)-f15625
    • 0xf15625
    • 0xf15625
    • 0xf15625

Raster Images from a URL

  • dummyimage.com — Set a dimension/hexCode color and generate a PNG from a URL: https://dummyimage.com/10x10/f15425/f15425.png. DummyImage.com also returns a JPEG or GIF.
  • http://ipsumimage.appspot.com/10x10,f15625?l=+ — Set a dimension,hexCode and alternatively annotate with ?l=text+and+use+plus+as+space. Example URL: http://ipsumimage.appspot.com/10x10,f15625?l=+
  • https://ipsumimage.appspot.com/10x10,f15625?l=+ — Set a dimension,hexCode and alternatively annotate with ?l=text+and+use+plus+as+space. See https://ipsumimage.appspot.com

SQLite3

DB Browser for SQLite is an open source, freeware visual tool used to create, design and edit SQLite database files.

  • brew install --cask db-browser-for-sqlite


Example: Inspecting cache.db for ambient cache of tiles. image


Docker

"Save yourself from installing the wrong thing."

docker pull osgeo/gdal:alpine-normal-latest
docker pull jekyll/jekyll:latest
docker pull maptiler/tileserver-gl:latest
docker pull klokantech/tileserver-gl:latest
docker pull roblabs/gdal

tileserver-gl

Example, Start tileserver-gl with current folder, port=8081, copy home URL to clipboard

  • alias s='docker run --rm -it -v "$(pwd)":/data -p 8081:80 klokantech/tileserver-gl --verbose; echo 'http://localhost:8081' | pbcopy'

git

One of the original block chains.

git config

# CLI help
git help config

# where your settings are located
less ~/.gitconfig

# local repo config
git config --list

# global config
git config --list --global

Updating Branches

# Update Git branches from master
git fetch
git rebase origin/master
# Update Git branch from master
git checkout master
git pull
git checkout <your-branch>
git merge master

Update Git fork from master

Review

git remote --verbose
git remote add upstream https://github.com/$aUser/$aRepo.git  # Example:  aUser=mapbox; aRepo=mapboxcli
git remote --verbose
git fetch upstream

# Check out your fork's local master branch.
git checkout master

# Merge the changes from upstream/master into your local master branch.
git merge upstream/master
# Submodules
git submodule add <repo> <destination_folder>

Git + log

View the commit history, but format it in your style.

git log --abbrev-commit --pretty=oneline
# ed9459e Update readme.md

git log --pretty=oneline
# ed9459e6329471274c3083dc824b767e45cc9 Update readme.md

git log --pretty=format:"%h %ad - %s" --date=format:'%b %d, %Y'
# 9264dbb Nov 04, 2019 - Update readme.md

git log --pretty=format:"* %ad - %s" --date=format:'%b %d, %Y'
# * Nov 04, 2019 - Update readme.md

Git + Stash

Use git stash when you want to record the current state of the working directory and the index, but want to go back to a clean working directory.

git stash list
# stash@{1}: On some_branch: branchit
# stash@{2}: On testflight: shipit
# stash@{3}: On dev: workit

# show the latest, or `0`th stash
git stash show -p stash@{0}

# save the patch to disk
git stash show -p stash@{1} > branchit.diff

Git Repos

export REPO=mapbox/mapbox-gl-native

git clone \
  https://github.com/$REPO.git \
  tmp/$REPO

cd tmp/$REPO

GitHub

Configuring Docker for use with GitHub Packages

echo $USERNAME
cat ~/.GITHUB_ACTIONS_TOKEN | docker login https://docker.pkg.github.com -u USERNAME --password-stdin

# docker tag IMAGE_ID docker.pkg.github.com/OWNER/REPOSITORY/IMAGE_NAME:VERSION
# docker push docker.pkg.github.com/OWNER/REPOSITORY/IMAGE_NAME:VERSION
docker tag 906f32a99d32 docker.pkg.github.com/roblabs/docker-qgis-ubuntu/qgis:ubuntu
docker push docker.pkg.github.com/roblabs/docker-qgis-ubuntu/qgis:ubuntu

Geospatial Data Abstraction Library (GDAL)

Thanks to the FOSS4G community and OSGEO.org

GDAL in docker

  • https://hub.docker.com/r/roblabs/gdal
    • docker pull roblabs/gdal
    • docker pull osgeo/gdal:alpine-normal-latest
    • # Step 1, set your GDAL_DOCKER_IMAGE
    • # GDAL_DOCKER_IMAGE=roblabs/gdal
    • GDAL_DOCKER_IMAGE=osgeo/gdal:alpine-normal-latest
  • docker run -v $(pwd):/data roblabs/gdal ogr2ogr
  • docker run -v $(pwd):/data roblabs/gdal gdalwarp
  • docker run -v $(pwd):/data roblabs/gdal gdal_translate
  • docker run -v $(pwd):/data roblabs/gdal gdal2tiles.py
  • docker run -v $(pwd):/data roblabs/gdal gdalinfo test.tif

Python modules

  • pip
  • mb-utileasy_install mbutil
  • rasteriosudo pip install rasterio
  • mapboxclipip install --user mapboxcli
    • You'll then need to include ~/.local/bin in your $PATH, export PATH=~/.local/bin:$PATH

Brew modules

  • https://brew.sh - The missing package manager for macOS
  • Awesome Brew tools in one line
    • brew install awscli cocoapods exif gifify git-gui ImageMagick jsonnet mapbox/cli/mapbox mupdf cocoapods tippecanoe tree webp wget
    • brew analytics off
  • aws - brew install awscli — Official Amazon AWS command-line interface
  • exif - brew install exif — Read, write, modify, and display EXIF data on the command-line
  • gcloud version - brew cask install google-cloud-sdk — manage resources and applications hosted on Google Cloud Platform
  • git gui - brew install git-gui
  • ImageMagick - brew install ImageMagick — Tools and libraries to manipulate images in many formats
    • montage -mode concatenate -tile 2x2 *.png -border 10 out.png
    • convert -size 1024x1024 -delay 150 -loop 0 *.png output.gif
    • convert -size 1024x1024 -quality 100 -delay 150 *.png video.mpg
  • jsonnet - brew install jsonnet — Domain specific configuration language for defining JSON data
  • mapbox command line interface - brew install mapbox/cli/mapbox
  • mutool - brew cask install xquartz; brew install mupdf - all purpose tool for dealing with PDF files
  • cocoapods - brew install cocoapods
  • gdal brew tap osgeo/osgeo4mac - brew install gdal2-pdf brew install gdal2-python
  • webp - brew install webp - Image format providing lossless and lossy compression for web images
  • wget - brew install wget - Internet file retriever
    • wget -P tmp/ https://roblabs/file.bmp # save files to PREFIX/..
    • wget https://roblabs.com/favicon.ico
  • tippecanoe - brew install tippecanoe - Build vector tilesets from collections of GeoJSON features
  • tree - brew install tree — tree -f . - Display directories as trees (with optional color/HTML output)
tree -f . -L 1
.
├── ./Desktop
├── ./Documents
├── ./Downloads
├── ./Library
├── ./Movies
├── ./Music
├── ./Pictures
├── ./Public

Java

  • AWS Corretto JDK
    • brew install --cask corretto
    • brew install --cask corretto@11.0.9.12.1

Ruby

Ruby, RVM & Gem notes

install from brew

# Install Ruby based dependencies via `brew`
brew list --versions ruby@2.6 || brew install ruby@2.6

# set the executables path
export RUBY_BREW=$(brew --prefix ruby@2.6)
echo ruby=${RUBY_BREW}

export GEM_DIR=$(${RUBY_BREW}/bin/gem environment gemdir)
echo GEM_DIR=${GEM_DIR}

# Update PATH
export PATH=${RUBY_BREW}/bin:$PATH
export PATH=${GEM_DIR}/bin:$PATH

# conditionally install dependencies
jazzy -version     || gem install jazzy
xcpretty --version || gem install xcpretty

# list the version that Ruby & Gem thinks is intalled
gem list jazzy xcpretty
# execute the Gem and list its version
jazzy -version
xcpretty --version

install from RVM

  • Install from rvm.io
  • curl -sSL https://get.rvm.io | bash -s stable
    • Add to path
      • source /Users/roblabs/.rvm/scripts/rvm
      • export GEM_HOME=~/.gem
      • export GEM_PATH=~/.gem

Node

Node and NPM notes

  • Node.js, which installs node and npm
npm install --global    \
  topojson-client  topojson-server \
  carthage              \
  csv2geojson           \
  xml-json              \
  @here/cli             \
  geojson-join          \
  @mapbox/geojson-merge \
  geojson-precision     \
  geojson-random        \
  geojson2csv-cli       \
  @mapbox/geojsonhint   \
  geojsonio-cli         \
  geojson-cli-bbox      \
  json                  \
  join-json             \
  prettier              \
  serve                 \
  @tmcw/togeojson-cli

Make a Basic GeoJSON Parser in Node

  • npm init -y - Create an empty package.json
  • Add these to package.json
    • "main": "index.js"
    • "dependencies": { "geojson-precision": "^1.0.0", "minimist": "^1.0.0", "concat-stream": "^2.0.0" }
  • npm install # install dependencies locally
  • Add the Javascript below to a new file called index.js
  • Usage:
    • geojson-random 2 | node index.js
    • geojson-random 2 > out; node index.js out

File listing: index.js

var fs = require('fs')
var argv = require('minimist')(process.argv.slice(2))  // parse argument options
var concat = require('concat-stream')
var geojsonprecision = require("geojson-precision")
var stdin
var usage = "usage info"
var result = { "type": "FeatureCollection", "features": [] };

if (argv.help || argv.h) {
  console.log(usage)
  process.exit()
}

if (argv._[0] && argv._[0] !== '-') {
  stdin = fs.createReadStream(argv._[0])
} else if (!process.stdin.isTTY || argv._[0] === '-') {
  stdin = process.stdin
} else {
  console.log(usage)
  process.exit(1)
}

// buffer all input
stdin.pipe(
  concat( function (buffer) {
    try {
        var geojson = JSON.parse(buffer)
      } catch (e) {
        return console.error(e)
    }

    // Process the input
    // Example 1:  Output what was input
    console.log(JSON.stringify(geojson));

    // Example 2:  output each feature element
   geojson.features.forEach(element => console.log(element.geometry.type));

   // Example 3:  trim precision to 6 digits
   console.log(JSON.stringify( geojsonprecision(geojson, 6)) );
  })
)

json

  • npm install --global json
  • echo '{"fred":{"age":42}}' | json fred.age
    • Pretty
    • apm install atom-beautify
    • npm install -g prettier
      • prettier is a great way to reformat GeoJSON coordinates & properties to a single line
      • "coordinates": [-171.94140039338671, 59.642861762832055]
      • geojson-random 3 | prettier --parser json --print-width 80

jsonnet

csv2geojson or geojson2csv

gifify

  • npm install -g gifify
  • gifify pct.mov -o pct.gif
  • gifify pct.mov -o pct.gif --from 45 --to 50 --speed 2 --resize 640:-1
  • docker - docker run -it --rm -v $(pwd):/data maxogden/gifify pct.mov -o pct.gif

geojson-random

geojson-random 3
geojson-random 3 > a.json
geojson-random 3 > b.json

geojsonio

geojsonio a.json
geojson-random 100 | geojsonio

geojson-merge

geojson-merge a.json b.json
geojson-merge a.json b.json | prettier --parser json

geojson-precision

geojson-random 3 > a.json
geojson-precision a.json b.json

join-json

echo { \"id\" : 3 } > a.json
geojson-random 3 > b.json
join-json -i a.json -i b.json -o out.json -f

geojsonhint

geojson-join

geojson-join test/against.json \
    --againstField=id \
    --geojsonField=id < test/random.geojson

jsontogeojson

d3 tools from mbostock

shp2json

ndjson

d3-dsv

Web servers

Editors

  • atom.io
    • apm list
    • apm install atom-beautify ds-ignore file-icons language-jsonnet markdown-image-assistant pretty-json prettier-atom terminus
# Open up keymap editor in Atom
# atom ~/.atom/keymap.cson
'atom-text-editor':
  'shift-cmd-p': 'pretty-json:prettify'
  'shift-cmd-m': 'pretty-json:minify'

Fonts, Glyphs, SVG

macOS

macOS Symbols

Symbols are useful in Text Replacements when typing text in any app in iOS & macOS. For example, you can create text replacements so you can "automatically replace certain text with other text or symbols." For example, when you configure you text replacements, you can type :330 to get 🕞.

Basic macOS symbols:

  • Caps Lock
  • Command
  • Control or Ctrl
  • Delete
  • ⏏︎ Eject
  • ⎋ Escape
  • ⎈ Helm
  • Option or Alt
  • Return

Finder

  • New Terminal Tab at Folder

    • Shift-Command (⇧⌘)-J
      • System Preferences > Keyboard > Shortcuts > Services
  • Show Hidden Files & Folders in Finder

    • defaults write com.apple.finder AppleShowAllFiles TRUE; killall Finder
    • Likely fixed in the Catalina era
  • Change folder of where Screenshots are saved, instead of Desktop

    • Fixed in the Catalina era with Grab.app
    • Shift-Command (⇧⌘)-3 for entire screen
    • Shift-Command (⇧⌘)-4 for a selected portion
    • defaults write com.apple.screencapture location ~/Downloads
    • killall SystemUIServer
  • macOS Hot Corners

macOS-hot-Corners

system_profiler

system_profiler -- reports system hardware and software configuration.


system_profiler -json SPDeveloperToolsDataType

system_profiler -listDataTypes
system_profiler SPPrintersDataType
system_profiler SPApplicationsDataType | grep Info
system_profiler SPNetworkDataType

sw_vers -- print Mac OS X operating system version information

sw_vers
sw_vers -productVersion

macOS Network Commands

Useful for turning on/off Ethernet or Wi-Fi in a script.

Turn on/off Ethernet

# Display services with corresponding port and device in order they are tried for connecting to a network.
networksetup -listnetworkserviceorder

# When an interface is marked `down`,
# the system will not attempt to transmit messages through that interface.
#
# This may be used to enable an interface after an `ifconfig down`
# sudo ifconfig <interface> down | up

sudo ifconfig en0 down
sudo ifconfig en0 up

Set Wi-Fi power

networksetup -setairportpower en1 off
networksetup -setairportpower en1 on

Xcode

xcodebuild

Build Xcode projects from the command line. You can also use xcodebuild to list the Schemes & Target that are in you Xcode Workspace or Project.

xcodebuild -version

# online help
man xcodebuild

# List schemes & targets
xcodebuild -list

# Example, to build the iosapp sample from Mapbox GL
xcodebuild -workspace ios.xcworkspace -scheme iosapp
  • Showing build times in Xcode
    • defaults write com.apple.dt.Xcode ShowBuildOperationDuration -bool YES
  • Explicitly set your Xcode path, in case you have a Xcode 10 or 11 or a Beta installed
    • xcode-select -print-path
      • Print or change the path to the active developer directory. This directory controls which tools are used for the Xcode command line tools (for example, xcodebuild) as well as the BSD development commands (such as cc and make).

# Default path for Xcode, where App Store installs.  As of Jul 2021
# Xcode 12.5.1
# Build version 12E507
sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer/
/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
export BUILD=12.0.5.12050022

# Use for Xcode Betas.  As of Jul 2021
# Xcode 13.0
# Build version 13A5155e
sudo xcode-select --switch /Applications/Xcode-beta.app/Contents/Developer/
/Applications/Xcode-beta.app/Contents/Developer/usr/bin/xcodebuild -version
export BUILD=13.0.0.13000025

# Xcode 10.3
# Build version 10G8
# Use Xcode 10.3 for upgrading Swift 3 projects.  As of Jul 2021, only runs on Catalina or possibly early versions of Big Sur
sudo xcode-select --switch /Applications/Xcode_10.3/Xcode.app/Contents/Developer/
/Applications/Xcode_10.3/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
export BUILD=10.0.1.10010046

# Xcode 11.7
# Build version 11E801a
sudo xcode-select --switch /Applications/Xcode_11.7/Xcode.app/Contents/Developer/
/Applications/Xcode_11.7/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
export BUILD=11.0.3.11030032

# Xcode 12.4
# Build version 12D4e
# macOS Catalina only supports up to Xcode 12.4
sudo xcode-select --switch /Applications/Xcode_12.4/Xcode.app/Contents/Developer/
/Applications/Xcode_12.4/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
export BUILD=12.0.0.12000032

# Xcode 12.5.1
# Build version 12E507
# Does not install on Catalina.  
# Error dialog
#   You can’t use this version of the application “Xcode.app” with this version of macOS.
#   You have macOS 10.15.7. The application requires macOS 11.0 or later.
sudo xcode-select --switch /Applications/Xcode_12.5/Xcode.app/Contents/Developer/
/Applications/Xcode_12.5/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
export BUILD=12.0.5.12050022
  • xcodebuild -showsdks -json
  • List simulators. Must be in a folder with a project or workspace, or pass it in with -workspace
    • xcodebuild build -destination 'platform=iOS Simulator'
  • Let Xcode render your Markdown
    • cd <project>.xcodeproj or cd <workspace>.xcworkspace
    • edit .xcodesamplecode.plist with the following contents
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"> <array/> </plist>