bits and pieces

this is the part where i explain the things i've learned related to command line interface

i wrote about command line here. most of the things i anticipate appearing on this page are going to be me learning more about this.

i wrote about the basics of vim here. i don't know how one would edit things in the tilde-verse without using vim, nano, or emacs...

something about sshing in here

ssh? what? they never talked about this in those tutorials...

*reminder,i'm on a mac* In order to generate an SSH key, you need to open Terminal (my favorite way is to hold down command+space bar and start typing 'terminal' and opening it from there) type ` ssh-keygen -t rsa`. You'll need to enter your password. Default location is fine.

So terminal spits out some avant garde art for you. go back to your home directory with `cd`, then type `ls -a`. You should see a directory called .ssh. go into it by typing `cd .ssh`.

At the minimum, you should see 2 files: id_rsa and id_rsa.pub. Congrats! You've got an SSH key pair!

great. thanks, shanx. i've got this thing now... what is it?

SSH stands for Secure Shell. When you would like to take control of a computer/server from far away (or across the room; who am i to judge?), one way to do so is to SSH in.

When you generated the SSH key pair, it was like creating a lock and key. id_rsa.pub is your "public key" and it works like a lock. You can give that to anyone, because all they can do is put a lock on stuff- cool, no big deal.

Sample places you might use your public key:

• GitHub, or another online code repository
• Heroku, Azure, OpenShift, or another platform as a service (PaaS) application
• tilde.club, tilde.town, or another super awesome pseudo-social unix server filled with amazing stuff from spectacular people!

too long with just text; time for an image.

Your id_rsa, however, is you "private key" and is the real key which unlocks your public key lock. DO NOT SHARE YOUR PRIVATE KEY WITH ANYONE, NOT EVEN YOUR MOM AND ESPECIALLY NOT YOUR CAT! Putting you public key everywhere is fine, because all it does is block other people from messing with it. But if they have your private key they can get in to ANYPLACE you have your lock.

oh man, i totally just used <strong> on you. i'm sorry- i don't want to freak you out, but it's kinda important. Why should you use SSH? Well it's pretty safe. We're not just talking "Password123" here. These are 2 matched sets of... hell, let's check them out!

Get back to the .ssh file if you moved away (`cd ~/.ssh`) and look at the file by typing `cat id_rsa` and `cat id_rsa.pub`. Yeah. try guessing that one.

now that your computer has this super secret private key and other useful places in the world have your pretty impressive public key, you don't have to remember a password. The computers trust each other, if you will. Any time you "shell in", the private key and the public key will do their secret handshake and unlock. i should avoid inventing metaphors at night.

how does this work in the tilde-verse?

so first you need to get a tilde account - tilde.town has a lovely form you can use. You'll request a name (like I am ~shanx) and you will provide your public key so they can "put a lock" on your part of the tilde server.

Once you have an account, you are going to SSH in! are you ready? In terminal, type `ssh your_account_name@tilde.town`.

Yup.

That's it.

Like Magic

images: nowhere near as easy as the tutorials had you believe

you want to pull a picture from your computer to your files in tilde town? you need the scp command (for "secure copy") format goes `scp ~/your/file/path/image.pjg name@tilde.town:~/desired_folder`

Note: you have to do this command from outside your ssh connection (I just open up another terminal window with command + n and type it there).

now that you've got a file, you need to allow the host server to find it: `chmod a+r image.jpg` and `chmod a+rx folder` will allow the server to show off your fabulous cat (or whatever image you want to show off)

Note: you have to do this command from inside the ssh connection.

"It's been said that 'graphical user interfaces make easy tasks easy, while command line interfaces make difficult tasks possible.'" ~William E. Shotts, Jr.

11/16 - TIL -Shell-ception?

I've been reading a lot, but I haven't gotten a lot that I fully grok, or at least understand enough that I feel I can write about it. So in the interest of not dropping off the face of the planet:

Today I ssh'd from my work laptop to my home laptop so that I could then ssh into tilde.town.

I went to System Preferences>Sharing and made sure the "Remote Login" box was checked. When the "Remote Login" option is selected, it will show you the IP to login to-- something along the lines of 'To log in to this computer remotely, type "ssh user_name@10.20.30.40".'

I then got out the other laptop, Opened terminal, typed that in, then shelled into 'town.

11/16 - TIL -PROCESSES

Ok, so I think this entry is gonna be a bit more academic. Oh Well! Here we go!

So every now and then everythign is terrible. You are reading this via some form of technology, so you are already aware of this. Knowing how to identify processes and services that are causign things to be terrible is the first step towards un-terrible-ization. Onward, we go~

When you turn on your computer, a program called init starts running. Init, in turn, starts up a whole slew of other programs and services (If you've ever heard of a 'parent process' and a 'child process', this is a good example of a parent- init- starting child processes). You also launch programs when you do something like opening up your browser. You can see what processes are running in any given directory by tyoing `ps`. In all likelihood, you will see one to a dozen results with the following headers:

• PID- "process ID" - it's like a program's social secuity number. When it gets added to the system, it gets the next number (init's PID is 1, for example).
• TTY - "teletype" - this lets you know which terminal session started the process.
• TIME - how much time did the process take to run?
• CMD - "command" - what program/service/app is running this?

`ps` only shows what is running in the current directory-- if you would like to see everythign that is going on, you could type `ps x`. There is a newheader here, STAT, which tells you what the programm is doing:

What? That's not enough for you? Fine, try `ps aux`-- you'll get everything from `ps x` PLUS:

• USER - who owns this process?
• %CPU - how much CPU is it usung
• %MEM - how much memory is it using?
• VSZ - virtual memory size (don't worry, I didn't know either, until I read this)
• RSS - "Resident Set Size" how much RAM it's using
• STARTED - time/day that process began

There's also `top`, which will give you a summary and live data, but I'm not sure I want to tackle that one yet. If you do try it, you will have to press control + c to exit out of that view.

KITTEN BREAK!

To open a process like xlogo (sorry i can't point you to a list of processes or anything), you would type `xlogo`.

Bam! After a bit, there is a window with a big black x in it! Cool, leave that there and go back intot the terminal window. What's that? You don't have control of the prompt? Oops! That's because xlogo is running just like a command, and you can't do anything else until it's done.

Ways you can regain control:

• quit xlogo by clicking the red 'close' circle or holding command + q
• quit xlogo from terminal by holding control + c
• stop (but don't quit) xlogo from terminal by holding control + z

If you stopped xlogo with control + z, you will notice that if you try to change the size of the window, the 'x' will stay the same size. That's because it's stopped. Terminal will have told us:

[1]+ Stopped xlogo

If we watned to launch xlogo to be in the background from the very start, we could type `xlogo &`.

getting violent on your processes

So let's say you ran `ps aux` or `top` and discovered a terrible beast that needed to be slain.

Find the PID of the process and run `kill PID`, replacing 'PID' with the number.

There are extra flags you can add to your `kill`, which you can see by typing `kill -l`. You can reference those either by a dash + the number, or a dash plus the part after "SIG".

handy kill flags:

• -9 / -KILL -- instead of sending the kill commmand to the process itself, this tells it's parent process to kill it-- use it as a last resort
• -15 / -TERM -- the default flag; if a process can still act on input, this tells it to shut down-- the same as control +c

• -18 / -CONT -- tells a process to continue after it's been stopped
• -20 / -TSTP -- terminal stop; same as control + z, it tells the program to stop
• -19 / -STOP -- tells the parent process to stop the process

ok, I'm about to fall over from all these commands

ONE LAST ONE

if you've got more than one instance of a process going you opened like 7 xlogos for somereason) you don't have to kill them one by one, you can get them all with `killall xlogo`

11/16 - TIL - MY FIRST SHELL SCRIPT

I know, I know, I'm just too damn fancy :)

Actually, it turned out that writing a shell script was pretty simple after coming this far. So `find` and `sed` have been pretty handy. And I had a bunch that I wanted to run (I added links to the top on the left! Which means I also needed to add anchors to the top). Turns out a shell script is really just a file with several cli commands in it!

So to keep everything nice and tidy, we'll make a home for our scripts: `mkdir ~/scripts`

Then you write a file and put each command you want to run on a new line. Here's mine:

Then you save is with a '.sh' extension and just call it with `./scriptname.sh` and it does its magic!

apparently if you run `export PATH="$PATH:~/scripts"` first, you won't have to add the `./` beforehand, but that didn't work for me ¯\(°_o)/¯

Also apaprently, terminal/bash will fail in silent but deadly ways unless you put certain precautions in place, so I now have a template that loks like this:

shout out to ~robbymeals for pointing out the preventative maintence blog mentioned above

11/15 - TIL - Investigating

I'm not really even sure how to describe the things I've been learning. OH WELL! This is me trying, anyways!

My good friend/coworker, Finch, was helping me un-f*#% the mess that is trying to use both Homebrew and Macports as package managers. At one point he started to explain... well, let's see if I can explain it, too...

Terminal/Bash/Linux treates everything like a file. Including the commands that you type in. So when you say `ls`, it gets written to the file "stdin", which sytands for "standard input". Your computer reads that file, does its magic, and then it writes its answer to a file called "stdout" (standard output) and then prints out the stdout file.

There's one other type of file that is used when you are working in Terminal: "stderr" (standard error). This file gets written to when the computer magic is broken, and also prints to the screen.

If he had stopped there I would have discarded that knowledge as trivia, but then he was like "Let's DO something with this!"

"Instead of just putting all of stdout and stderr into terminal, let's have it write to foo.txt. Type in `chown shanx /usr/local/something &> foo.txt`"

This time when we ran it, we didn't see the errors. Then, we ran `cat foo.txt` and there they were! Here's how that happened:

• `chown` is a command to 'change owner', which Homebrew needed us to do to certain files if it was ever going to work.
• `shanx` was who should be the new owner
• `usr/local/something` stands in for the directory that shanx need to be the owner of
• `&>` says 'push both stdout and stderr to a file' (if it were just `&`, it would only push stdout; if it were `2&`, it would only push stderr)
• `foo.txt` is the file that &> will push to.
• `cat` pretty much says 'read this', but can be used in so many different ways that I'll probably have to come back to it

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

PIPING

so remember exciting time I learned about `find` and `sed`? That command had a | in it, which is called a pipe. A pipe basically says "and then...". In the case of that TIL, it took the results of `find` AND THEN... used that output for `sed`.

There's lots of other things you can use with pipes, though:

• `(command) | sort > foo.txt` will put all the output from (command), sort it alphabetically, and write it to foo.txt
• `(command) | sort | uniq > foo.txt` will sort, then remove any duplicate files (leaving only unique files) and write to foo.txt
• `(command) | grep awesome` will run (command), but will only show you things that contain the word 'awesome'!

`grep` is one of those commands that I love because I like finding words for some reason.

If I were to run `grep TIL cli.html`, it would print out all of the lines in this file that have "TIL" in them.

If I only want to look at the top part of a file, I can use `head -n 12 foo.txt`

• `head` means top part of a file-- to read the bottom part instead, I would have used `tail`
• `-n 12` says how many line to show (in this case, 12) If you leave this part blank, the default is 10 lines
• `foo.txt` says what file to read from.

dhgwilliam showed me a neat trick using `head` which I'm going to write about on vim page!

11/11 - TIL - More About Wildcards

I have been a little bit terrible about explaining wildcards to anyone. It occurs to me that some folks might not know about this kind of searching, and in all liklihood, if they are familiar, it's probably in a different syntax. So. An explanation here, and an update in the reference table

why wildcards?

It's one thing to search for a single file (I love Spotlight on Mac, and how it is the easiest keyboard shortcut: command + spacebar). But what about when you want to find groups of things and lots of files that match a specific type?

In command line, we can use wildcards to look for patterns in files and do things to files that match those patterns. For example, I have a directory where I keep the images I am going to load to tilde.town. In an effort to reduce file size, I edited them and saved them in a different format. I now had 2 versions of each picture in that file- the orginal .png image, and the smaller .jpg image.

I knew I wasn't going to use the .png files anymore (a version was stored elsewhere) and I decided I didn't want them cluttering up the directory. I could have clicked each .png file and sent it to the trash from Finder, OR I could do this:

`rm *.png`

Wildcards like `*`, stand in for something. `*` stands for "anything", so `*cats` will return anything that ends in "cats", like tigercats, tomcats, cats, and fatcats. By running `rm *.png`, I told my computer to remove all the files (in the current directory) that have anything followed by .png.

useful wildcards

The wildcard `?` stands for "any one character". So `B?b` would find Bob, Bib, and Brb, but not Blab, since there is more than one character between B and b. You can use several ?s in a row- to find pictures taken on New Year's Day, you might look for `2014-01-01??.??.??.png` (if your camera auto-titles pictures date+time.png)

You can search for several characters by enclosing them in square brackets- `[bch]at*` will return anything that begins with bat, cat, or hat. You can even use a range, like [a-f] (note: It's case sensitive, so that would only find lower case a through f - use [a-fA-F] to cover upper & lower case). For digits, you can use a range like [5-7] to find 5, 6, or 7, or [0-9] to look for any single digit.

Similar to [0-9], you could also use what is called a Character Class- these are also enclosed in square brackets, so it looks like double the fun. `version[[:digit:]].html` will return files version1.html, version2.html, etc... Check out several other character classes in the reference table

`!` - the exclamation point is called "bang" in computer-ese. No clue why, it just is. What it stands for, is usually "not". So `[!Z]` would return everything except files that begin with Z.

wildcards in the wild

I'm in the directory where all my photos get uploaded with the format YYYY-MM-DDhr.mn.sc.jpg (for example: 2012-10-07 15.16.19.jpg). I run `cp 2013*.jpg ~/Desktop/Pics_from_2013` to put a copy of all the photos from 2013 in a directory called Pics_from_2013 on my desktop.

Maybe I just need some lunch, but these are tough to come up with... I hope to fill this part in more usefully later...

11/11 part deux -

this isn't so much a thing I learned as it is a command I stole off StackOverflow that I repurposed and it worked.

`for f in *.png; do mv "$f" "${f/2012-10/oct}"; done` took a bunch of picture files, and if they began with "2012-10", it replaced that part with "oct".

i will be a very happy camper when i get to the point where i can come up with this stuff on my own

11/11 part trois -

`echo` - I've seen this used to literally echo back strings (enter `echo bunny men` and it will return "bunny men"). But the reason `echo *` returns all the things in the directory instead of just "*" is because... order of operations? Much like 1+1*3=4 and not 6, command line parses the wildcards I talk about above before it parses `echo`... I think. This will make more sense one day, I suspect.

11/11 part... I'm sorry; I son't actually know French

What a day of learning! There I was getting some mad `git rebase` practice, when all of a sudden the default git editor was like "Hey, I heard you want to write this file- would you like to Toggle the use of DOS format? What about Mac format? appending? prepending? Toggle backing up of the original file?!?!" and I'm all like "What does this even meeeeeeeannnnnn?!

So it turns out, that was Nano, the text editor that usually gets the friendlier billing of the *nix-y crew. Without being able to say `:wq`, I began to flounder. Luckily, datagrok, with infinite wizardry revealed the magic incantaion `git config --global core.editor vim`, so now I don't have to figure out what all that Toggle nonsense was about... at least for now...

11/10 - Today I Was Reminded: If at first you don't succeed, try `sudo`.

11/9 - TIL:` find . -name "*.html" -print0 | xargs -0 sed -i '' -e 's/beehive.png/beehive.jpg/g'`

Boy howdy, is it a mouthfull. I put pictures on the server (I really should just get a flikr or something), and I kept maxxing out my disk quota. Rather than ask for more space (again), I decided I would see if I could work with smaller file sizes.

Locally, I copied the images, shrank them down to 75%, and then saved them as JPEG, rather than PNG. Massive space savings.

Over on the tilde.town server, I removed the original images (`rm file_name`), and locally ran `scp` with the new files.

I had 2 choices. Go in to each html file and replace the images, or... auto-magically make things happen. Here's what the command means:

• `find .` -translates to "look for files in . (which is shorthand for 'this directory') and any sub-directories"
• `-name "*.html" - translates to "whose Name matches the pattern *.html (`*` means "anything", so look for any file named anything that ends in .html)"
• `-print0 ` - is a weird one, but basically means "sort and store the results in a way where things won't get screwed up if someone put spaces or new lines in the file names"
• `|` - translates to "after you do the thing on my left, do the thing on my right"
• `xargs` - is short for "extended arguments" - it basically pulls the results from the `find` and feeds them to `sed`
• `-0` - remember when we said `-print0`? Here, `-0` tells `xargs` where one results stops and the next begins, based on that.
• `sed` - stands for "Stream Editor" - this is so cool- it messes with words! And you tell it how to mess with them by passing it options and arguments like those below:
• `-i ''` - the `i` stands for "In Place" as in, don't put this in a different directory, and the empty apostrophes say "I don't need no stinkin' Backup Copy!"
• -e 's/beehive.png/beehive.jpg/g'` - the `e` says "add the following to the list of things you are doing", and if you've been playing with vim you might notice that find and replace command is looking for the old name of the file and replacing it with the new name of the file.