What is this?

This is a ___

What is a ___?

This is a ___

Codelabs

My goal is to complete these 8 codelabs in order become more familiar with Flutter and Material Design in order to eventually build extraordinary apps that improve people’s lives.

I chose Flutter because:

I discovered a series of YouTube videos called Flutter Widget of the Week and have been watching a couple of them daily.

Write Your First Flutter App, part 1

In this codelab I followed the directions and made an app which suggests potential names for a startup company.

I tested it on both iPhone and Android simulators and on an actual Android phone connected to my laptop with a usb cord.

Screen recording of the app running in a simulator

I made a diagram to understand the hierarchy of widgets.

Widget Hierarchy

This is the code which generates the app:

import 'package:flutter/material.dart';
import 'package:english_words/english_words.dart';

void main() => runApp(new MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      title: 'Startup Name Generator',
      home: new RandomWords(),
    );
  }
}

class RandomWords extends StatefulWidget {
  @override
  RandomWordsState createState() => new RandomWordsState();
}

class RandomWordsState extends State<RandomWords> {
  final List<WordPair> _suggestions = <WordPair>[];
  final TextStyle _biggerFont = const TextStyle(fontSize: 18.0);

  @override
  Widget build(BuildContext context) {
    return new Scaffold(
      appBar: new AppBar(
        title: const Text('Startup Name Generator'),
      ),
      body: _buildSuggestions(),
    );
  }

  Widget _buildSuggestions() {
    return new ListView.builder(
        padding: const EdgeInsets.all(16.0),
        itemBuilder: (BuildContext _context, int i) {
          if (i.isOdd) {
            return const Divider();
          }
          final int index = i ~/ 2;
          if (index >= _suggestions.length) {
            _suggestions.addAll(generateWordPairs().take(10));
          }
          return _buildRow(_suggestions[index]);
        });
  }

  Widget _buildRow(WordPair pair) {
    return new ListTile(
      title: new Text(
        pair.asPascalCase,
        style: _biggerFont,
      ),
    );
  }
}

Write Your First Flutter App, part 2

In this codelab I continued to develop the Startup Name Generator app by allowing a user to select their favorite names and adding a new page which shows the selected names.

Saving names and viewing saved names

When a user touches the icon in the upper right they go to the saved names page. In other words, (Flutter words), the navigator pushes the new route onto the route stack.

Widget Hierarchy

This is the code which generates the app:

import 'package:flutter/material.dart';
import 'package:english_words/english_words.dart';

void main() => runApp(new MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      title: 'Startup Name Generator',
      theme: ThemeData.dark(),
      home: new RandomWords(),
    );
  }
}

class RandomWordsState extends State<RandomWords> {
  final List<WordPair> _suggestions = <WordPair>[];
  final Set<WordPair> _saved = new Set<WordPair>();
  final TextStyle _biggerFont = const TextStyle(fontSize: 18.0);

  @override
  Widget build(BuildContext context) {
    return new Scaffold(
      appBar: new AppBar(
        title: new Text('Startup Name Generator'),
        actions: <Widget>[
          new IconButton(icon: const Icon(Icons.list), onPressed: _pushSaved),
        ],
      ),
      body: _buildSuggestions(),
    );
  }

  Widget _buildSuggestions() {
    return new ListView.builder(
        padding: const EdgeInsets.all(16.0),
        itemBuilder: (BuildContext _context, int i) {
          if (i.isOdd) {
            return new Divider();
          }
          final int index = i ~/ 2;
          if (index >= _suggestions.length) {
            _suggestions.addAll(generateWordPairs().take(10));
          }
          return _buildRow(_suggestions[index]);
        });
  }

  Widget _buildRow(WordPair pair) {
    final bool alreadySaved = _saved.contains(pair);
    return new ListTile(
        title: new Text(
          pair.asPascalCase,
          style: _biggerFont,
        ),
        trailing: new Icon(
          alreadySaved ? Icons.favorite : Icons.favorite_border,
          color: alreadySaved ? Colors.red : null,
        ),
        onTap: () {
          setState(() {
            if (alreadySaved) {
              _saved.remove(pair);
            } else {
              _saved.add(pair);
            }
          });
        });
  }

  void _pushSaved() {
    Navigator.of(context).push(
      new MaterialPageRoute<void>(
        builder: (BuildContext context) {
          final Iterable<ListTile> tiles = _saved.map(
            (WordPair pair) {
              return new ListTile(
                title: new Text(
                  pair.asPascalCase,
                  style: _biggerFont,
                ),
              );
            },
          );
          final List<Widget> divided = ListTile.divideTiles(
            context: context,
            tiles: tiles,
          ).toList();
          return new Scaffold(
            appBar: new AppBar(
              title: const Text('Saved Suggestions'),
            ),
            body: new ListView(children: divided),
          );
        },
      ),
    );
  }
}

class RandomWords extends StatefulWidget {
  @override
  RandomWordsState createState() => new RandomWordsState();
}

Building Beautiful UIs with Flutter

In this codelab I made an interface for a chat app called Friendlychat.

This app includes animation and widgets with more structure than those in the previous codelab.

Sending a message on the Friendlychat app

Widget Hierarchy of Friendlychat

import 'package:flutter/material.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/cupertino.dart';

void main() {
  runApp(new FriendlychatApp());
}

class FriendlychatApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      title: "Friendlychat",
      theme: defaultTargetPlatform == TargetPlatform.iOS
          ? kIOSTheme
          : kDefaultTheme,
      home: new ChatScreen(),
    );
  }
}

class ChatScreen extends StatefulWidget {
  @override
  State createState() => new ChatScreenState();
}

class ChatScreenState extends State<ChatScreen> with TickerProviderStateMixin {
  final List<ChatMessage> _messages = <ChatMessage>[];
  final TextEditingController _textController = new TextEditingController();
  bool _isComposing = false;

  void _handleSubmitted(String text) {
    _textController.clear();
    setState(() {
      _isComposing = false;
    });
    ChatMessage message = new ChatMessage(
      text: text,
      animationController: new AnimationController(
        duration: new Duration(milliseconds: 900),
        vsync: this,
      ),
    );
    setState(() {
      _messages.insert(0, message);
    });
    message.animationController.forward();
  }

  Widget _buildTextComposer() {
    return new IconTheme(
        data: new IconThemeData(color: Theme.of(context).accentColor),
        child: new Container(
            margin: const EdgeInsets.symmetric(horizontal: 8.0),
            child: new Row(children: <Widget>[
              new Flexible(
                child: new TextField(
                  controller: _textController,
                  onChanged: (String text) {
                    setState(() {
                      _isComposing = text.length > 0;
                    });
                  },
                  onSubmitted: _handleSubmitted,
                  decoration:
                      new InputDecoration.collapsed(hintText: "Send a message"),
                ),
              ),
              new Container(
                margin: new EdgeInsets.symmetric(horizontal: 4.0),
                child: Theme.of(context).platform == TargetPlatform.iOS
                    ? new CupertinoButton(
                        child: new Text("Send"),
                        onPressed: _isComposing
                            ? () => _handleSubmitted(_textController.text)
                            : null,
                      )
                    : new IconButton(
                        icon: new Icon(Icons.send),
                        onPressed: _isComposing
                            ? () => _handleSubmitted(_textController.text)
                            : null,
                      ),
              ),
            ])));
  }

  @override
  Widget build(BuildContext context) {
    return new Scaffold(
      appBar: new AppBar(
        title: new Text("Friendlychat"),
        elevation: Theme.of(context).platform == TargetPlatform.iOS ? 0.0 : 4.0,
      ),
      body: new Container(
          child: new Column(
            children: <Widget>[
              new Flexible(
                child: new ListView.builder(
                  padding: new EdgeInsets.all(8.0),
                  reverse: true,
                  itemBuilder: (_, int index) => _messages[index],
                  itemCount: _messages.length,
                ),
              ),
              new Divider(height: 1.0),
              new Container(
                decoration:
                    new BoxDecoration(color: Theme.of(context).cardColor),
                child: _buildTextComposer(),
              ),
            ],
          ),
          decoration: Theme.of(context).platform == TargetPlatform.iOS
              ? new BoxDecoration(
                  border: new Border(
                    top: new BorderSide(color: Colors.grey[200]),
                  ),
                )
              : null),
    );
  }

  @override
  void dispose() {
    for (ChatMessage message in _messages)
      message.animationController.dispose();
    super.dispose();
  }
}

class ChatMessage extends StatelessWidget {
  ChatMessage({this.text, this.animationController});
  final String text;
  final AnimationController animationController;
  @override
  Widget build(BuildContext context) {
    return new SizeTransition(
        sizeFactor: new CurvedAnimation(
            parent: animationController, curve: Curves.elasticOut),
        axisAlignment: 0.0,
        child: new Container(
          margin: const EdgeInsets.symmetric(vertical: 10.0),
          child: new Row(
            crossAxisAlignment: CrossAxisAlignment.start,
            children: <Widget>[
              new Container(
                margin: const EdgeInsets.only(right: 16.0),
                child: new CircleAvatar(child: new Text(_name[0])),
              ),
              new Expanded(
                child: new Column(
                  crossAxisAlignment: CrossAxisAlignment.start,
                  children: <Widget>[
                    new Text(_name, style: Theme.of(context).textTheme.subhead),
                    new Container(
                      margin: const EdgeInsets.only(top: 5.0),
                      child: new Text(text),
                    ),
                  ],
                ),
              ),
            ],
          ),
        ));
  }
}

const String _name = "Jason";

final ThemeData kIOSTheme = new ThemeData(
  primarySwatch: Colors.orange,
  primaryColor: Colors.grey[100],
  primaryColorBrightness: Brightness.light,
);

final ThemeData kDefaultTheme = new ThemeData(
  primarySwatch: Colors.brown,
  accentColor: Colors.blueGrey,
);

Firebase for Flutter

In this codelab I made an app which lets people vote for their favorite baby names.

The big thing that this app has which the previous one didn’t is a database, which is where vote counts are stored. This is a big step up since it allows data to be shared between users. It’s kind of like the app changed from being just an isolated app on a single phone to being something “out there in the cloud” that many phones can connect to.

I’ve worked with databases and cloud databases before and this one worked seamlessly. I’m looking forward to using Firebase databases more.

The animation shows the cloud database updating in realtime as I vote for a name from an iPhone simulator.

Cloud database updating in realtime as I vote for a name in the app

Widget Hierarchy

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Baby Names',
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() {
    return _MyHomePageState();
  }
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Baby Name Votes')),
      body: _buildBody(context),
    );
  }

  Widget _buildBody(BuildContext context) {
    return StreamBuilder<QuerySnapshot>(
      stream: Firestore.instance.collection('baby').snapshots(),
      builder: (context, snapshot) {
        if (!snapshot.hasData) return LinearProgressIndicator();

        return _buildList(context, snapshot.data.documents);
      },
    );
  }

  Widget _buildList(BuildContext context, List<DocumentSnapshot> snapshot) {
    return ListView(
      padding: const EdgeInsets.only(top: 20.0),
      children: snapshot.map((data) => _buildListItem(context, data)).toList(),
    );
  }

  Widget _buildListItem(BuildContext context, DocumentSnapshot data) {
    final record = Record.fromSnapshot(data);

    return Padding(
      key: ValueKey(record.name),
      padding: const EdgeInsets.symmetric(horizontal: 16.0, vertical: 8.0),
      child: Container(
        decoration: BoxDecoration(
          border: Border.all(color: Colors.grey),
          borderRadius: BorderRadius.circular(5.0),
        ),
        child: ListTile(
          title: Text(record.name),
          trailing: Text(record.votes.toString()),
          onTap: () => Firestore.instance.runTransaction((transaction) async {
                final freshSnapshot = await transaction.get(record.reference);
                final fresh = Record.fromSnapshot(freshSnapshot);

                await transaction
                    .update(record.reference, {'votes': fresh.votes + 1});
              }),
        ),
      ),
    );
  }
}

class Record {
  final String name;
  final int votes;
  final DocumentReference reference;

  Record.fromMap(Map<String, dynamic> map, {this.reference})
      : assert(map['name'] != null),
        assert(map['votes'] != null),
        name = map['name'],
        votes = map['votes'];

  Record.fromSnapshot(DocumentSnapshot snapshot)
      : this.fromMap(snapshot.data, reference: snapshot.reference);

  @override
  String toString() => "Record<$name:$votes>";
}

Hacker Tools

Hacker Tools is a course taught by MIT students. It gives a broad overview of tools which are available to a hacker (a computer programmer). It is divided into these sections:

Each section has a lecture video, notes, and exercises.

I chose this course because I have used many of the tools covered and I’d like to be able to use them better.

Virtual Machines and Containers

Video, notes, exercises

Virtual machines and containers are different ways of running simulated computers on a computer.

I used a Windows 10 virtual machine to check and see if this website works on the Microsoft Edge web browser. I found that the animated png images do not animate but just show the first frame. Animated gif images work.

'This website in Microsoft Edge inside a Windows virtual machine. The images are not animated.'

Exercises

Virtual Machines

Download and install a hypervisor.

I installed VirtualBox which I had used before.

Create a new virtual machine and install a Linux distribution (e.g. Debian).

I tried out some operating systems including Windows and versions of Linux.

Screen recording of a Windows 10 virtual machine running on my Apple laptopScreen recording of a CentOS (Linux) virtual machine running on my Apple laptop

Experiment with snapshots. Try things that you’ve always wanted to try, like running sudo rm -rf --no-preserve-root /, and see if you can recover easily.

I tried running the suggested command, sudo rm -rf --no-preserve-root /, on a Debian virtual machine. It caused the screen to go blank and the virtual machine to become unresponsive. I was able to restore the machine back to its original state using the snapshot I created before running the command.

Read what a fork-bomb (:(){ :|:& };:) is and run it on the VM to see that the resource isolation (CPU, Memory, &c) works.

I read part of the fork bomb Wikipedia entry. A fork bomb is a process which replicates itself twice. The result is that each of the replicas replicate and eventually there are so many processes running that the system is overloaded.

I ran the fork bomb command on a virtual machine. A couple of seconds later the terminal was flooded with error messages and the virtual machine became unresponsive. My laptop was not affected. In fact, I am typing this while the fork-bomb is still exploding in the virtual machine.

Screen recording of a fork bomb on a Linux virtual machine

Install guest addons and experiment with different windowing modes, file sharing, and other features.

I installed guest addons on a virtual machine and was able to drag and drop files, cut and paste, and share a folder between machines.

Using drag and drop to move a file from my desktop to the virtual machine's desktop

Containers

Choose a container software (Docker, LXC, …) and install a simple Linux image. Try SSHing into it.

I already had docker installed on my computer from another project. I was able to start a ubuntu linux image and connect to its shell with the command docker run -it ubuntu bash.

Starting a linux image and typing the ls command into its shell

Search and download a prebuilt container image for a popular web server (nginx, apache, …)

I followed the instructions on the nginx Docker Official Image site to run an nginx webserver from a prebuilt image.

An nginx webserver from a prebuilt image

The instructions point out that a better way of doing this is to rebuild the image using a two line dockerfile.

FROM nginx
COPY website /usr/share/nginx/html

An image of an nginx webserver is created from a dockerfile and then run connected to local port 8080

Shell and Scripting

Video, notes, exercises

Using the shell is an important skill and I am continually improving my ability.

Exercises

If you are completely new to the shell you may want to read a more comprehensive guide about it such as BashGuide. If you want a more indepth introduction The Linux Command Line is a good resource.

These are both good resources.

PATH, which, type

We briefly discussed that the PATH environment variable is used to locate the programs that you run through the command line. Let’s explore that a little further

Run echo $PATH (or echo $PATH | tr -s ':' '\n' for pretty printing) and examine its contents, what locations are listed?

Folders named bin or sbin in various locations. The folder names are short for “binaries” and “system binaries”. I read some interesting things about sbin here.

The command which locates a program in the user PATH. Try running which for common commands like echo, ls or mv. Note that which is a bit limited since it does not understand shell aliases. Try running type and command -v for those same commands. How is the output different?

Commands which, type, and command -v all gave the same information in slightly different formats.

Running which, type, and command -v for some common commands

Run PATH= and try running the previous commands again, some work and some don’t, can you figure out why?

The output for builtin and aliased commands was unchanged. The others output a “not found” message or nothing at all.

Special Variables

What does the variable ~ expands as? What about .? And ..?

~ is the user’s home directory.

. is the current directory.

.. is the parent directory.

What does the variable $? do?

It holds the exit code of the previous command. It is 0 if there was no error.

What does the variable $_ do?

It holds the last argument of the previous command.

What does the variable !! expand to? What about !!*? And !l?

!! expands to the previous command.

!!* expands to all the arguments of the previous command.

!l expands to the last command which starts with l.

Look for documentation for these options and familiarize yourself with them

I found information about these options at these three sites.

xargs

Sometimes piping doesn’t quite work because the command being piped into does not expect the newline separated format. For example file command tells you properties of the file.

Try running ls | file and ls | xargs file. What is xargs doing?

The command file isn’t expecting input from STDIN; It takes the files as arguments. So piping with ls | file doesn’t work.

xargs takes the lines which are passed to its STDIN from ls through the pipe (|) and passes them to file as arguments.

The result is that the command ls | xargs file displays the file types of files in the current directory.

Shebang

When you write a script you can specify to your shell what interpreter should be used to interpret the script by using a shebang line. Write a script called hello with the following contentsmake it executable with chmod +x hello. Then execute it with ./hello. Then remove the first line and execute it again? How is the shell using that first line?

  #! /usr/bin/python

  print("Hello World!")

It printed “Hello World!” but when I removed the first line it resulted in a syntax error message. The first line tells the shell which program should be used to interpret the script.

You will often see programs that have a shebang that looks like #! usr/bin/env bash. This is a more portable solution with it own set of advantages and disadvantages. How is env different from which? What environment vairable does env use to decide what program to run?

env uses the $PATH environment variable to decide which program to run.

which can find any command available to you at the command line.

Pipes, process substitution, subshell

Create a script called slow_seq.sh with the following contents and do chmod +x slow_seq.sh to make it executable.

  #! /usr/bin/env bash

  for i in $(seq 1 10); do
          echo $i;
          sleep 1;
  done

There is a way in which pipes (and process substitution) differ from using subshell execution, i.e. $(). Run the following commands and observe the differences:

./slow_seq.sh | grep -P "[3-6]"

grep -P "[3-6]" <(./slow_seq.sh)

echo $(./slow_seq.sh) | grep -P "[3-6]"

Observing the differences between pipes, process substitution, and subshell execution

With pipes (|) and process substitution (<()) the output is passed line by line in real time to the next step.

With subshell execution ($()), the process is completed and then the entire output is passed along.

Misc

Try running touch {a,b}{a,b} then ls what did appear?

Four files: aa, ab, ba, and bb.

Sometimes you want to keep STDIN and still pipe it to a file. Try running echo HELLO | tee hello.txt

This prints HELLO and saves HELLO in the file hello.txt.

Try running cat hello.txt > hello.txt what do you expect to happen? What does happen?

I expect the contents of file hello.txt to be written to file hello.txt. Actually, the file is now empty.

Run echo HELLO > hello.txt and then run echo WORLD >> hello.txt. What are the contents of hello.txt? How is > different from >>?

Experimenting with > and >>

hello.txt contains the characters HELLO\nWORLD.

>> appends to a file while > overwrites it.

Run printf "\e[38;5;81mfoo\e[0m\n". How was the output different? If you want to know more, search for ANSI color escape sequences.

Experimenting with the color of printf's output

The sequence \e[38;5;81m determines the color of the output of the printf command. I found useful information about ANSI color escape sequences here.

Run touch a.txt then run ^txt^log what did bash do for you? In the same vein, run fc. What does it do?

Experimenting with caret substitution

It entered the previous command, substituting txt with log.

Keyboard shortcuts

As with any application you use frequently is worth familiarising yourself with its keyboard shortcuts. Type the following ones and try figuring out what they do and in what scenarios it might be convenient knowing about them. For some of them it might be easier searching online about what they do. (remember that ^X means pressing Ctrl+X)

^A, ^E

Move cursor to beginning or end of line.

^R

Search history.

^L

Clear screen.

^C, ^\ and ^D

Kill foreground process by sending SIGINT (^C) or SIGQUIT (^\). ^D closes the shell by sending EOF.

^U and ^Y

Cut or paste line.

Command-Line Environment

Video, notes, exercises.

I learned a lot in this section and I took some notes of things which were new to me. I ended up installing many new tools including trash, tldr, atool, fd, tmux, fzf, and ack.

Things I didn’t know before:

\ls or command ls will ignore the alias

/tmp is cleared on reboot so it’s a good place to put temporary files

To troubleshoot globbing use echo Example: echo **/*

** will catch any subtree /b/c/

In zsh you can expand a glob by pressing tab

If you type a command and a dash and press tab it will tell you all the possible flags grep -

autojump or fasd or ranger lets you navigate the filesystem faster

fd is faster and better than find

rg can search within files

fzf is grep in real time

tldr more compact man pages

aunpack for tar without flags

tmux is a terminal multiplexer that allows you detach and attach to remote shell sessions without stopping running processes

Exercises

Run cat .bash_history | sort | uniq -c | sort -rn | head -n 10 (or cat .zhistory | sort | uniq -c | sort -rn | head -n 10 for zsh) to get top 10 most used commands and consider writing shorter aliases for them

The ten most used commands in my terminal

Choose a terminal emulator and figure out how to change the following properties:

I have been using iTerm2 but I chose to try out Alacritty because it has gpu acceleration and is supposed to be the fastet terminal emulator. I appreciate snappy interfaces.

I was able to change the font style, color, and scrollback history size by editing the file ~/.config/alacritty/alacritty.yml.

It looks like the standard scheme has 256 colors. This is probably so that colors can be stored in 8 bit memory slots. (There are 256 possible ways to configure 8 switches).

Install fasd or some similar software and write a bash/zsh function called v that performs fuzzy matching on the passed arguments and opens up the top result in your editor of choice. Then, modify it so that if there are multiple matches you can select them with fzf.

I installed fasd but it doesn’t work for me. I think it’s conflicting with some oh-my-zsh setting. I struggled with it for a while and gave up for now.

Since fzf is quite convenient for performing fuzzy searches and the shell history is quite prone to those kind of searches, investigate how to bind fzf to ^R. You can find some info here

I was able to set up the ^R keybinding by running /usr/local/opt/fzf/install which I learned here. Now I can use fuzzy search to quickly find commands in my history.

Fuzzy searching through history using fzf with the ^R keybinding

What does the --bar option do in ack?

It shows an ascii art version of beloved Star Wars character Admiral Ackbar.

Output of command ack --bar

Data Wrangling

Video, notes, exercises.

Installed st, gnuplot, rename

Things I didn’t know before:

sed streaming line editor uses regular expressions

sed -E means you don’t need all the \'s

sed regular expressions are greedy - they keep trying to find matches

/1 /2 replace matched capture groups

uniq -c picks out unique lines and counts

awk is a line editor that knows about fields

counts can be piped into statistics programs like R or gnuplot

st gives simple statistics

- can mean stdin

usually you end up using grep | awk | paste

Exercises

If you are not familiar with Regular Expressions here is a short interactive tutorial that covers most of the basics

I’m already familiar with regular expressions.

How is sed s/REGEX/SUBSTITUTION/g different from the regular sed? What about /I or /m?

The /g (global) flag tells sed to replace all occurances, not just the first.

/I means “ignore case”; Upper and lowercase characters are not differentiated.

/m stands for “multiline” and it tells sed that ^ and $ mean the beginning and end of a line, not the beginning and end of the entire string.

To do in-place substitution it is quite tempting to do something like sed s/REGEX/SUBSTITUTION/ input.txt > input.txt. However this is a bad idea, why? Is this particular to sed?

I tried it and it erased the contents of input.txt. This is not particular to sed since I saw the same thing happen in an exercise in the Shell and Scripting section with the command cat hello.txt > hello.txt.

I finally found the answer here. The file is opened in truncate (overwrite) mode before it is read. This erases the contents.

The answer also points out that it is risky to read and write to the same file like this, even with >> which appends and does not clear the file. If the buffer which holds the contents as they are being modified is not large enough to hold the entire file, data which has been appended could be read, processed, and appended again, resulting in a continually growing file.

For some reason it reminds me of trying to swap the values of two variables. It requires a temporary third variable, as the javascript code below suggests.

var a = 1;
var b = 2;
var c;

c = b;
b = a;
a = c;

They can actually be swapped like this: [a, b] = [b, a], but this a wrapper. The little piece of memory which stores a variable is really just a bunch of switches and can only hold one configuration at a time, so a third variable is required.

Implement a simple grep equivalent tool in a language you are familiar with using regex. If you want the output to be color highlighted like grep is, search for ANSI color escape sequences.

I decided to use node.js. The script should accept lines from its STDIN and print to its STDOUT only lines which include the search string. I called it findLinesWith.

Using my script to search the shell history

#! /usr/bin/env node

const searchString = process.argv[2];

if (!searchString) {
  console.log('You must enter a search string as the argument');
  return;
}

const readline = require('readline');

const greenCode = '\033[31m';
const defaultCode = '\033[39m';

const interface = readline.createInterface({
  input: process.stdin,
});

interface.on('line', (line) => {
  if (line.includes(searchString)) {

    highlightedLine = line.replace(
      new RegExp(searchString, 'g'),
      greenCode + '$&' + defaultCode
    );

    process.stdout.write(highlightedLine);
    process.stdout.write('\n');
  }
});

There is a nice breakdown of the color codes here.

Sometimes some operations like renaming files can be tricky with raw commands like mv . rename is a nifty tool to achieve this and has a sed-like syntax. Try creating a bunch of files with spaces in their names and use rename to replace them with underscores.

Using "rename" to replace spaces with underscores in file names

Look for boot messages that are not shared between your past three reboots (see journalctl's -b flag). You may want to just mash all the boot logs together in a single file, as that may make things easier.

I used a Linux computer for this one since I wasn’t able to find an equivalent way to access boot logs on my mac.

After trying journalctl --list-boots I found that the system was only keeping track of the most recent boot. I was able to change this using some information I found here.

First, for each of the past three reboots I put the log into a file, removing the first line (column labels) and the first three columns (time/date info) using the following command. I also removed any duplicate lines from the same boot which is necessary for the next step.

for i in {0..2}; do journalctl -b -$i | tail -n +2 | sed -E 's/Feb [0-9]+ [0-9]+:[0-9]+:[0-9]+ localhost.localdomain (.*)/\1/' | sort | uniq > bootlog$1; done

Then I made use of the fact that the set of lines which are not shared by all three files is the same as the union of the sets of files which are not shared by any two files. In other words, I found the lines which are not shared by files 1 and 2, the lines which are not shared by files 2 and 3, and the lines which are not shared by files 3 and 1, and then combined them all. I used the following command.

{ cat bootlog0 bootlog1 | sort | uniq -u & cat bootlog1 bootlog2 | sort | uniq -u & cat bootlog2 bootlog0 | sort | uniq -u; } | sort | uniq > notshared

There were 1063 messages which weren’t shared between all three boot logs. Compare this number of messages in the boot logs: 1242, 1605, and 1415. I noticed that there were still triplicates since process id’s and timestamps were sometimes mixed into the messages. I was able to remove some of them a brought the count down to 604 with this command:

cat notshared | sed -E 'x/\[[0-9\.]+\]/\[\]/g' | sort | uniq > notshared2

Produce some statistics of your system boot time over the last ten boots using the log timestamp of the messages

Logs begin at ...

and

systemd[577]: Startup finished in ...

I extracted the start and end times for the last ten boots, used date to convert them to seconds, expr to calculate the differences, then fed them all into st for statistics. The average boot time was 2 minutes and 11 seconds.

#! /usr/bin/etc bash

for i in {0..9}
do
  starttime=$(date --date "$(journalctl -b -$i | sed -n '2p' | awk '{print $1,$2,$3}')" +%s)
  endtime=$(date --date "$(journalctl -b -$i | grep 'Startup finished' | awk '{print $1,$2,$3}')" +%s)
  difference=$(expr $endtime - $starttime)
  echo $difference >> boottimes
done

st boottimes
N	min	max	sum	mean	stddev
10	113	141	1307	130.7	8.90755

Using gnuplot to view the last ten boot times

Find the number of words (in /usr/share/dict/words) that contain at least three as and don’t have a 's ending. What are the three most common last two letters of those words? sed's y command, or the tr program, may help you with case insensitivity. How many of those two-letter combinations are there? And for a challenge: which combinations do not occur?

I used the following command to discover that there are 7023 words wich meet the criteria.

cat /usr/share/dict/words | sed -nE "/.*a.*a.*a.*/ p" | sed -E "/('s)$/ d" | tr "[:lower:]" "[:upper:]" | sort | uniq | wc -l

Of these 7023 words, the three most common last two letters are “al”, “ia”, and “an”, which I found using:

cat /usr/share/dict/words | sed -nE "/.*a.*a.*a.*/ p" | sed -E "/('s)$/ d" | tr "[:lower:]" "[:upper:]" | sort | uniq | sed -E "s/.*(..)$/\1/" | sort | uniq -c | sort -nr | head -n 3

The following command revealed that there are 150 different two letter endings which occur in these 7023 words.

cat /usr/share/dict/words | sed -nE "/.*a.*a.*a.*/ p" | sed -E "/('s)$/ d" | tr "[:lower:]" "[:upper:]" | sort | uniq | sed -E "s/.*(..)$/\1/" | sort | uniq -c | sort -nr | wc -l

Since there are 26 possible letters, there are 26 times 26, or 676 possible two letter combinations. 150 of them appeared in our list which leaves 526 combinations which did not occur.

Find an online data set like this one or this one. Maybe another one from here. Fetch it using curl and extract out just two columns of numerical data. If you’re fetching HTML data, pup might be helpful. For JSON data, try jq. Find the min and max of one column in a single command, and the sum of the difference between the two columns in another.

I used the Dow Jones Index Data Set. The data is in csv (comma separated values) format. I extracted the opening and closing prices into a file.

I found the min and max opening prices with the following command.

awk -F ',' 'FNR == 1 {next} {print substr($4,2)}' dow_jones_index.data | sort | tee >(head -n 1) | tail -n 1

I found the sum of the difference between the two columns with this command:

paste -sd+ <(awk -F ',' 'FNR == 1 {next} {openprice=substr($4,2) ; closeprice=substr($7,2) ; print openprice - closeprice}' dow_jones_index.data) | bc -l