Developing for developers (Part One)

Setting up for dev

Some nights I wake up with great ideas, at least they seem great in my half slumber daze. I tend to scribble down these sleep-induced brain farts on my scratch pad that is with me at all times.

Having ideas is great, taking the time to work through the idea and truly assess its value is tedious, grueling, boring as fuck, but most importantly vital in allowing a developer to really understand the benefits and potential uses of the "Idea".

So... I got the idea, yay for me!
I worked out the bugs and have put together an action plan.
Lance 2... World 0!

Now its time to revolutionize!!

And go!!..., no wait, hold on I need a place to actually bring this to life. A playground or development area that has no bounds.

"Drum roll please..."
Introducing the "dev environment".

While this may not seem a topic that can be overlooked, not having a "ready to go" dev environment will dramatically slow down the pace of getting this idea out, and I can tell you from personal experience has been the leading factor to shelving many of my past ideas.

So for any serious entrepreneur, developer or ideas person, having a pop-up dev environment is critical.

The toolbox

One of the most critical assets to a developer is their toolbox, the dev environment is the most critical tool in that box... ok, enough blabbing, let's figure out how to get this bad boy built.

There are 3 basic requirements:

1. Shouldn't cost me a dime or as cheap as possible
2. Quick to go up, Quick to go down
3. Don't fuck up my computer (this should probably be number one)

Firstly though, while I do want you to get this tool built and running as quickly as possible, I also want to enable you, by letting you know of the few failures I myself have experienced.

One note before I start talking about the failures. Each person has their own preferences and failure to me might be a success for someone else, so don't judge me and I won't tell you to shove it up your ass judge you.

Some options

Wamp/Xampp

These were the tools I used in the earlier days, they fitted my requirements as a beginner. I had no idea how to setup anything so clicking the install button and having everything work was a dream.

Until I required some additional PHP libraries. Im not going to get into what this entailed, but effectively I spent more time sorting out my environment areas than did actual coding.

Total bummer!! Wamp/Xampp, it was fun while it lasted... but to be honest you just don't give me that hard-on like you used to.

Cloud Services

In truth I have used 2 cloud services

• Amazon
• Digital Ocean

Amazon offers a great free tier program with there smaller servers, once you understand the vast amount of services they offer, it is really easy going.

Digital Ocean is truly a simple platform designed for developers, where you can spin up a virtual server instance or droplet as they term it, install what you require (assuming you have the knowledge) and you are running. When you are done, you close the virtual space and move on with your life.
An added benefit is that Digital Ocean have put out a shit load of tutorials on setting up many different environments, so pros to them!

Lance's Way

My system, just to give a brief summary, is the coming together of two similar and complementary systems, the merging of the strength of Superman and the Wealth of Batman; The power of the Hulk and the speed of the Flash; It really is the PB and J of dev environments.

Ok, ok, enough fooling around, Vagrant using VirtualBox combined with docker and docker-compose. See I needed all those synonyms else that would have been the world's shortest paragraph!

Why you do this guy?

It fits all of my requirements!

1. Free to use!
2. Easy up, easy down. no, really the commands are:
1. vagrant up (Server goes up)
2. docker-compose up -d (Docker containers are running)
3. docker-compose down (Docker containers are not running)
4. vagrant halt (Server is turned off)
5. vagrant destroy (Server is destroyed)
3. Encapsulated environment means I'm not installing tons of weird shit on my machine, so less of a chance of this fucking my system.
4. Bonus Reason! If your idea really is good and you bring on another developer, you simply share your vagrant file with them and BOOM!!! They coding quicker than Usain Bolt in the 100 Meter dash.

Putting it together

Use a fucking version control system (vcs)! No seriously, I don't care if its SVN, Mercurial or Git, just use one. You'll thank me later.

Personally I suggest git and I would advise using Bitbucket. I don't have anything against Github and I even have an active Github account, but bitbucket simply offers free private repositories (as many as you want). This means you can have the ease of using version control, and the ease of not forking out wads of cash.

Now that you have your VCS up, you will need two separate repositories, one for your vagrant project and another for your docker project. Of course, any additional development will be put in separate repositories.

Vagrant + Virtual Box

Install Virtual Box (from here)

Install Vagrant (from here)

Both the installers are pretty straight forward for Windows and Mac, so don't be a dick and just install them.

Vagrant Setup

There are tons of tutorials on setting up your vagrant machine, so I'm not going to go through each piece, but rather share with you the vagrant file that I use.

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  
  config.vm.box = "geerlingguy/centos7"
  config.vm.box_version = "1.1.3"

  # Vagrant SSH
  config.vm.network :forwarded_port, id: 'ssh', guest: 22, host: 2221
  

  # Many ports to ssh into docker containers

  for i on 2001..2030

      config.vm.network :forwarded_port, guest: i, host: i

  end

  
  # Nginx HTTP/S ports
  config.vm.network :forwarded_port, guest: 8080, host: 8080
  config.vm.network :forwarded_port, guest: 9080, host: 80

  config.vm.network :forwarded_port, guest: 9443, host: 443
config.vm.provider "virtualbox" do |vb|
    vb.name = "my_vm"
    vb.customize ["modifyvm", :id, "--memory", 2048]
  end

  config.vm.provision "shell", path: "base_provision.sh"

end

Note the very last line, this is a bash file that you can edit in order to install whatever libraries you may need on the vagrant server. Remember though that no development is done on this server, it is simply a space to run your docker containers.


And that's it for part one, in the next part I will discuss installing and setting up docker, building docker images, using docker-compose and some other generally useful tools for your dev environment.

What happened to my disk space

For a while now I’ve been using docker and docker containers on Amazon ECS for all of my
dev projects. Docker lets me focus on the actual project rather than wasting time on the
structure of the server (I’ll be discussing docker in greater detail in an upcoming article).

Docker is by no means a silver bullet and, like all services, there are some issues. The one
that I began to encounter was in the way the service carries out its logging. It’s a problem
that only affects ECS and resulted in many a sleepless night for me.

In order to preserve the logs and make them printable with docker commands, you need to
store the logs as a json file. The issue here is that the default log rotation prevents docker
from being able to read the log file and so after a relatively short period of time this log file
uses up all of your disk space.

If you’re wondering if there is a log rotate option that will work with docker, you would be
correct, however I haven't had the chance yet to test it and so for this article I’m going to
focus on the method I use to handle the problem.

When I first encountered the disk space issue I needed to hunt down the culprit that was
responsible for it. So I went the usual route and used the du command coupled with sort to
list the usages from lowest to highest.

du / | sort -nr

Ideally my culprit should have been the last entry in the list. Unfortunately, seeing as I was
using close to 100% of the available hard drive space, the sort wasn’t able to work.

I therefore needed a different method and, after some searching, I came across someone
with the same issue on Stack Exchange who had found a solution. Win!

The answer is to use a grep and filter out any files that are not bigger than 1G

du -h / | grep '[0-9\.]\+G'

This will print out the culprit which you can then delete and get back in business.

My next step is to figure out whether the log rotation function will work with docker so that
I won’t have to do this every so often.

References:

stackexchange - Tracking down where disk space has gone on Linux?
man du - du(1)