Code Smell: Solution Explosion

Symptoms

  • You’ve got more than one (Visual Studio) solution depending on the outputs from another.
  • Your checking binaries (.dlls, etc.) that aren’t third party into your version control system, as you do with third party stuff.

Causes

  • You have clumps of projects that are highly cohesive. You feel you need to organize them. A solution seems like a good organizational unit above projects.
  • More than one team is working on dependants. To avoid integration pain, they want to work on just their code, using a static version of the dependency (a compiled branch.)

What to Do

  • Put all the projects into one solution.
  • Use folders to organize them, instead of solutions.
  • If there are multiple teams, they all work on their projects in the same solution.
  • If it takes too long to build, unload projects you’re not changing right now. Load them again or use a command line build before checking in or doing cross project refactorings.

Payoff

  • Reduction in build and version control complexity
  • Reduction in code ownership
  • Less big-bang when updating dependencies.
  • More “we’re all in this together”
  • For web applications, you might end up using server resources far more efficiently if the modules can share an app pool. This can make a huge difference when you have a lot of projects.

Contraindications

  • If the project really, really needs to be complicated (say, 100+ screens/pages, each with 20+ controls/fields, or more than 15 developers) then it makes sense to separate them, at the same time spending effort to minimize coupling between the solutions. The idea is to make this more of a last resort. Rough estimate: solution/developer should be about 1/15.
Advertisements

Easiest Capistrano 3 Deployment Setup For Your Dedicated/VPS Server

In this post I will show you how to install Capistrano 3 for deployments to your dedicated/VPS server. Things are little bit different (and easier) when you deploy to the cloud (heroku), but clouds are expensive (up to 4 times in my case). So let’s play and have fun with setting all the things manually and save some money.

This post assumes that you have followed my previous articles on setting up Ruby/Rails production environment: How To Install RVM (Ruby Version Manager) and How to install Passenger for Nginx + few tricks

Here is the very simple diagram we gonna follow:

Dev -- Prod -- BitBucket

Dev is our development machine. It can look little bit different if you run it on Windows or on Mac (see my article Easy Rails with Vagrant + VirtualBox on Windows about how to run Ruby and Rails on Windows with Vagrant):

                                               
 +-------------------+      +-------------------+   
 |      Windows      |      |                   |   
 | +--------------+  |      |                   |   
 | |    linux     |  |      |        Mac        |   
 | |   running    |  |  or  |     Dev machine   |   
 | | in vagrant   |  |      |                   |   
 | +--------------+  |      |                   |   
 +-------------------+      +-------------------+   
                                                       

What you need to understand here is that you have to set the link between Dev and Prod. Prod is our production server. In case of running Windows with Linux and Vagrant, you have to set the link between the the Linux inside of Windows and production server. If you’re on Windows, we’ll run deployment process from this Linux box, keep this in mind. On Mac OS it is little bit easier.

Okay, but how to set the link up and what does it mean – “set the link between Dev and Prod”? It is easy, you just need to be able to log in from Dev to Prod by ssh without any passwords. Here is the command you can use to check if this link is already configured:


dev$ ssh 'deploy@forum.rubyschool.us'

Not that I’ll use forum.rubyschool.us for hostname here and fake IP of ‘144.76.1.222’, it’s the same. We’ll deploy the website ‘devhq.io’, which is located on the same host. And I’ll use “dev$” and “prod$” prefixes to show where command should be executed.

The command above should reply back with the error message, because the link is not set yet. So let’s do that. The first step is to create deploy user. Let’s create user for deployment and add it to sudo group.


prod$ sudo adduser deploy
prod$ sudo adduser deploy sudo

Then let’s edit sudoers file:


prod$ sudo nano /etc/sudoers

add change this line:


%sudo ALL=(ALL:ALL) ALL

to this:


%sudo ALL=(ALL) NOPASSWD:ALL

So all sudo users won’t need to type password for sudo commands. Passwordless sudo is required in some cases for capistrano. See capistrano docs for more. Here also you might want to take a look at sudoers file to see if deploy user is not mentioned there already. If it is, make sure he is passwordless.

You can check if sudoers file was updated correctly by executing:

user@prod$ su deploy # will ask the password, it's OK
deploy@prod$ sudo bash # should NOT ask the password
deploy@prod# exit

Now we have passwordless sudoed user with login ‘deploy’. It’s half the battle only. Passwordless sudoes means that used won’t be asked for password when s/he types “sudo something”. But when you try to login from Dev to Prod with ssh:


dev$ ssh deploy@forum.rubyschool.us

…you’ll be asked for the password. We don’t want that and we need passwordless login. Let’s copy ssh certificate from Dev to Prod for ‘deploy’ user. Check if you already have ssh certificate generated on your Dev for logged in user:


dev$ ls -la $HOME/.ssh/

You should see “id_rsa.pub” file listed. If it’s not there, it’s time to generate new one. Don’t type password when prompted, just leave it blank:


dev$ ssh-keygen

Great! Now you have your certificates generated on your Dev machine. Let’s copy public certificate to Prod so you won’t need to type password when logging in. Copying is easier with ssh-copy-id command:


dev$ ssh-copy-id deploy@forum.rubyschool.us

Now you can try to login from Dev to Prod with ssh command to check if it works. You should be able to that without a password:


dev$ ssh deploy@forum.rubyschool.us

Now Capistrano can use this feature for deployment purposes. Cool! What’s next?

Let’s think about another link on our diagram: Prod to Bitbucket (GitHub). Production server should have access to your code repository. Let’s check if you have this access already. Execute this command on your server, but under ‘deploy’ user(su deploy command). It’s important, because you may have access via ssh certificate from one user, and don’t have an access from another one.


deploy@prod$ git ls-remote --heads git@bitbucket.org:ro31337/devhq.git

If you don’t have git installed on your server, you can install it with:


prod$ sudo apt-get install git

But wait a second, where does this git link (git@bitbucket.org:ro31337/devhq.git) come from? Well, you can find it on your Bitbucket or Github settings page for your source code repository, or you can just simply check .git folder (it’s can be hidden) in your source code repository. In config file you’ll find url property to repository. The output for ‘git ls-remote… ‘ command is something like:


4ce15674cb4e0a0c12c726eb5e586e7655f880fb refs/heads/master

To be honest, you should get an error at this point, because the link between Prod and Bitbucket was not set. We’ve created ‘deploy’ user on Prod, but the certificate for newly created user is not present at Prod. Let’s check if you have one (run this command on production server, we have already installed certificates on development machine, but it is not Dev machine!):


prod$ ls -la $HOME/.ssh/

See if “id_rsa.pub” is there. If it’s not, run ssh-keygen command on Prod. Now we should have “id_rsa.pub” file created and it’s time to upload contents on this file to your Bitbucket (or Github). I usually do it by output the file contents to the console and copying it from there:


prod$ cat $HOME/.ssh/id_rsa.pub

Copy the output to Bitbuket/GitHub SSH global settings and click save. Test if this command works now:


deploy@prod$ git ls-remote --heads git@bitbucket.org:ro31337/devhq.git

At this point we should have all the links up on the way from Dev to Prod, and from Prod to BitBucket. And now the funny starts: setting up Capistrano, woohoo!

I’ll cover 3.4+ version here, it probably work with 3.x Capistrano branch. Keep in mind that there is a big difference between version 2 and 3. Let’s install Capistrano gem. Run on your Dev machine:


dev$ gem install capistrano

See the hint at end of this article if you have permissions issues here.

Well, gem is installed. Now let’s update our Gemfile and then execute Capistrano installation procedure. Add to the end of your Gemfile:

group :development do
  gem 'capistrano'
  gem 'capistrano-rails'
  gem 'capistrano-bundler'
  gem 'capistrano-passenger'
end

Not that I’ve included ‘capistrano-passenger’ gem, because my server runs with Passenger on Nginx. If you use Unicorn server, things in this tutorial won’t work for you. Run bundle install and execute Capistrano installation procedure:


dev$ bundle install
dev$ bundle exec cap install

Output for the last command:

mkdir -p config/deploy
create config/deploy.rb
create config/deploy/staging.rb
create config/deploy/production.rb
mkdir -p lib/capistrano/tasks
create Capfile
Capified

Great! We’ve been “capified” and we have a set of new files in the project. In very basic scenario you have to modify two of these files: production.rb and deploy.rb. The first one is very easy, I’ve just added one line in my production.rb:


server '144.76.1.222', user: 'deploy', roles: %w{web app}

Where ‘144.76.1.222’ is address of your server. Things are little bit tricky for deploy.rb, so I’ll just mention my uncommented ‘set’ statements here:

set :application, 'devhq'
set :repo_url, 'git@bitbucket.org:ro31337/devhq.git'
set :deploy_to, '/var/www/devhq.io'
set :log_level, :info
set :pty, true
set :linked_files, fetch(:linked_files, []).push('config/database.yml', 'config/secrets.yml')
set :passenger_restart_command, "touch #{release_path}/tmp/restart.txt"
set :passenger_restart_options, -> {}

deploy_to is directory where our website will be hosted. Create it now if you don’t have one:


prod$ mkdir -p /var/www/yourprojectname/

Before we execute the Capistrano deploy command (remember it now: bundle exec cap production deploy), I want you to know about Capistrano deployment features. There will be two top-level directories on your production server in /var/www/yourprojectname/, releases and shared:


releases
|- current
|- ...
shared

Releases directory will contain ‘current’ release and other releases directories (they will look like a datestamp: 20150507185726). Shared will keep the shared data between your releases in time, so this data won’t be overwritten by new deployments. It could be your config files with sensitive information: you probably don’t want to keep sensitive information in your source code repository. It could be any other data, like user uploads. So uploading sensitive files once and keeping them there sounds like a good idea. And we can keep database.yml and secrets.yml files in this directory.

The implementation behind this idea is very easy. On deployment, config/database.yml and config/secrets.yml will be symlinked to correspondent files in shared directory. Nothing fancy!

Now let’s run Capistrano and let it fail:


dev$ bundle exec cap production deploy

The output is


ERROR linked file /var/www/devhq.io/shared/config/database.yml does not exist on 144.76.1.222

It has failed because we don’t have any files in shared folder. Capistrano should create a symlink, but it can’t. Let’s upload these files. We could create additional Capistrano task for that, but looks like overhead for the simple demo. So instead of creating a task, let’s upload them with the scp command (assuming you’re located at project directory):


dev$ scp config/database.yml 'deploy@144.76.1.222:/var/www/devhq.io/shared/config/'
dev$ scp config/secrets.yml 'deploy@144.76.1.222:/var/www/devhq.io/shared/config/'

You should be okay now and your website should be deployed successfully. If not, keep on reading, I’ll show you some hints. You can get the following error on fresh Ubuntu installation:


ExecJS::RuntimeUnavailable: Could not find a JavaScript runtime

But it’s easy to resolve with:


prod$ sudo apt-get install nodejs

Another error you may have:


You are not authorized to query the status for this Phusion Passenger instance

Check if you have the following two lines in deploy.rb:


set :passenger_restart_command, "touch #{release_path}/tmp/restart.txt"
set :passenger_restart_options, -> {}

It’s an old way of restarting passenger without sudo permissions required.

If you have


sudo: no tty present and no askpass program specified

Make sure you have specified in your deploy.rb:


set :pty, true

If you have other errors, login under root and deploy user on Prod and make sure these commands work:


rvm current
rvm list
rvm use ruby-2.2.0 # your ruby version,

If you have something like ‘not found’ error, make sure you have installed and executed everything under these both users (deploy and root):


sudo apt-get install rvm

Try to run deployment process again:


dev$ bundle exec cap production deploy

It should work, but I also had fancy output for ‘rvm use ruby-2.2.0‘ command under deploy and root user:


RVM is not a function, selecting rubies with 'rvm use ...' will not work.

One-time solution is to run:


sudo apt-get install rvm
source ~/.rvm/scripts/rvm
type rvm | head -n 1
rvm requirements
rvm use ruby-2.2.0

But when you log out, your changes won’t be saved. To permanently resolve this, you should copy this line from .bash_history to .bashrc:


[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"

Finally, the last Capistrano command will finish with something like:


[0d37c034] Finished in 0.584 seconds with exit status 0 (successful).

Congratulations! Your app has been deployed.

Keep in mind that it was very simple deployment scenario, something just to start from. Hopefully, later I’ll cover more complicated scenarios. Good luck!

Useful commands:

rvm info # show environment and other RVM variables
rvm current # get the current ruby version installed
rvm list # list all rubies installed
rvm gemset list_all # list all gems installed

See also:

http://www.talkingquickly.co.uk/2014/01/deploying-rails-apps-to-a-vps-with-capistrano-v3/

http://corlewsolutions.com/articles/article-10-how-to-deploy-rails-applications-using-capistrano-3-1-and-windows-7

How to install Passenger for Nginx + few tricks

In this post I’ll cover how to install ‘passenger’ for nginx. What is passenger and why do you need it? Well, if you deploy your ruby applications straight to the cloud (heroku, etc), you don’t need that. But when it comes to running ruby apps on your own dedicated server (which I prefer), you need some magic to achieve this goal. If you don’t have your own dedicated/virtual server, you’ll probably won’t find this article very useful.

Before following instructions in this manual, make sure you have RVM installed (see my previous post). When you have RVM installed, you’ll need some sort of server to run your apps on. You can always use rails s -b 0.0.0.0 command, but built-in rails server is not production-ready and works well for development purposes only. So here the passenger comes into play.

Passenger itself is little bit tricky. The trickiest part of using Passenger and Nginx is that you can’t update your nginx – you have to compile, build new version, install, and remove old one. New version should be installed into the new directory.

OK, now the question “how”. Start with installing passenger gem:

gem install passenger

And then

rvmsudo passenger-install-nginx-module

The command above will run interactive and very friendly (greetings to Phusion team in Netherlands!) step-by-step guide where I just leave default values.

Now let’s check if everything’s installed:

cd /opt/nginx/
ls -l

Sample output:

drwxr-xr-x 2 root root 4096 May  4 23:33 conf
drwxr-xr-x 2 root root 4096 May  4 23:33 html
drwxr-xr-x 2 root root 4096 May  4 23:33 logs
drwxr-xr-x 2 root root 4096 May  4 23:33 sbin

One more thing I really recommend to do right now is to make a backup copy of config file:

cd /opt/nginx/conf/
cp nginx.conf nginx.conf.bak.1

If you have your old nginx installation running, make backups for your config files in /etc/nginx because now we’re going to uninstall previous version:

apt-get remove nginx nginx-full nginx-light nginx-naxsi nginx-common

Done? Great! Time to run nginx:

sudo /opt/nginx/sbin/nginx

Let’s try to get a page from localhost and show it on your screen to see if newly installed nginx is running:

wget -O - http://localhost

You should see something like “Welcome to nginx!” and some html code. Well, great! But how to stop it? Here is the trick:

sudo kill $(cat /opt/nginx/logs/nginx.pid)

Run wget command again and see that it has failed. Now we can run and stop nginx instances. What now? Everything works, but I’d love to get those “service nginx start”, “service nginx stop” commands back. Thanks to folks for posting this github gist. Here are few commands you need to replace init.d script:

cd /tmp
wget https://gist.githubusercontent.com/mwlang/3c7c3c832621d24051ae/raw/c8825bf2e9c9243201e4e0e974626501592ce81e/nginx.sh
sudo cp nginx.sh /etc/init.d/nginx
sudo chmod +x /etc/init.d/nginx
sudo update-rc.d nginx defaults

Now we can use “service nginx start”, “service nginx stop”, “service nginx stop”, etc. commands. Check yourself with wget-command above.

Now you can update nginx.conf with your previous configuration. When hosting ruby/rails apps, make sure you pointing them to public folder:

   server {
      listen 80;
      server_name www.yourhost.com;
      root /somewhere/public;   # <--- be sure to point to 'public'!
      passenger_enabled on;
   }

This command also helped when I messed everything and upgraded my ruby from 2.2.0 to 2.3.0:

rvmsudo passenger-install-nginx-module

Basically, it downloads and rebuilds your nginx with the settings for latest ruby in your system.

How to install RVM (Ruby Version Manager)

Here are few steps that I recommend to do on newly installed ubuntu machine. Let’s say you just installed rails-dev-box vagrant virtual machine, or you have fresh ubuntu installation – it doesn’t matter – developers use different versions of Ruby. So how to manage them?

Well, it was already mentioned a couple of thousand times, but things are little bit different over the years. In this article we’ll install Ruby Version Manager (RVM) for ubuntu. With few really minor modifications it should also work for Mac OS X.

For the sake of simplicity I usually run sudo bash command before. So you don’t need to run sudo something every time.

The first step – update a list of packages:

apt-get update

Next, I had no proper certificates on my fresh ubuntu machine for downloading RVM. Run this:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

After that we can download and install RVM itself:

curl -L get.rvm.io | bash -s stable

It’s “almost complete” according to the message. So you need to run few additional commands:

source /etc/profile.d/rvm.sh

And

rvm requirements

The last one will detect and download all RVM requirements. And finally we can install ruby binaries. The command itself is very easy:

rvm install VERSION

But you have to know what VERSION you want to install. If you want to install 2.2.0, just type:

rvm install 2.2.0

But how to know which version you need to install? Well, that’s easy. But there are few scenarios. How are you going to use the current virtual machine?

* If you planning to run rails app downloaded from someone’s Github page, just go application directory and locate .ruby-version file. You’ll see ruby version required to run the app.

* If you running your own app, look at your .ruby-version. If you don’t have one, run ruby --version command from your development shell. This is why it’s very important to have your .ruby-version for your app.

* If you don’t know, go to ruby downloads page and look for the “latest stable version is…”

Keep in mind with version manager you can install multiple ruby binaries. And only one of them you can make default. Go back to your shell and type:

rvm use 2.2.0 --default

And now you can install rails gem:

gem install rails

Try to log in as a normal user or type “exit” if you’re under the root user. Let’s check if you can install gems under the standard user:


ro@ubuntu:~$ gem install foreman
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the /usr/local/rvm/gems/ruby-2.2.0 directory.

If you have this error, just execute the command with path to directory above:


sudo chmod -R 777 /usr/local/rvm/gems/ruby-2.2.0

Good luck!

A few years ago, at a company whose name everyone here knows…

Source

A few years ago, at a company whose name everyone here knows I, an old gray-hair, almost lost my full-time job for warning a VP that she was about to make a mistake I’d seen before. I honestly thought she was joking when she told me about the approach she was going to use to deal with a “hard deadline” of four months. I laughed along with her joke until, with a shock, we both realized that I was actually laughing at her actual plan. She became indignant and ended our meeting (just the two of us), getting me transferred for “not being supportive of the team” and trying, unsuccessfully, to get me fired.

She gave the job to her three dozen “supportive” young programmers, who ended up taking 28 months (!!) to get the “4-month” project finally working.

So, after all that, did she eventually apologize and tell me that, yes, she’d made the same mistake I told her I’d seen before with the same results for the same reason, and that she should have at least asked me more about it instead of throwing me out? Of course not. This is the real world. She never spoke to me again and never forgave me.

Easy Rails with Vagrant + VirtualBox on Windows

There are plenty of posts describing on how to install Ruby and Rails on Windows. But there are two main approaches:

  • Install Ruby and Rails directly, I mentioned that before in Installing Ruby On Windows – The Right Way. The main disadvantage of this approach is that is limited to Ruby 1.9.x on Windows (well, last time I tried – two weeks ago). You can install Ruby 2.x+, but Rails won’t work. I don’t know why, but after googling for a couple of hours I found that it’s a common problem.
  • Second way is to install Vagrant + VirtualBox. Folks say it’s easy and the best way to go. I can’t say it isn’t the best way to go, but it’s not easy as I really expected. So in this post I’ll cover that approach so you guys hopefully can save a couple of hours and use them for the having fun with Ruby and Rails.

Why it should be easy, and why it’s not? Reason one – VirtualBox. It’s widely used, but it can refuse to work on your computer. If it doesn’t work, try to uninstall VirtualBox and Vagrant, and reinstall them again. Install VirtualBox, and then Vagrant. But let’s start from the beginning.

1. Go and grab VirtualBox. If it didn’t work yesterday, it can work today. Versions are updating pretty often. If it doesn’t work, try to rollback few versions back. Try to disable (uninstall) antiviruses. I have to admit that after lots of attempts I’m still unable to run it on one of my computers. I don’t know why, may be complete Windows reinstall can help. But it works pretty well on another computer, and I’m able to use it.

2. Install Vagrant. You can download it from the website http://www.vagrantup.com. “vagrant up” is the command you’ll use frequently to initialize your virtual machine.

3. Download rails dev box.  Don’t clone repository.  If you use msysgit for Windows, git can be configured to checkout files with CRLF endings and bash scripts won’t work in that case. So just download zip-file and unpack.

4. Switch to directory with Vagrantfile and run “vagrant up” (I use Far commander for file operations, if you don’t have any kind of file manager, you will need to run cmd.exe before this step). It will take some time do download and configure Ubuntu box.

It should go pretty well, but I had few BSODs on another because of VirtualBox. After that run “vagrant ssh” to check if you’re able to log on to linux machine. If everything’s fine, you’ll be able to see welcome message. You can type “exit” to exit from linux back to windows console (virtual machine will keep on running), but don’t do it now, we still have some things to do.

5. Install rails with command “sudo gem install rails”.

6. When rails is installed we can go and create our rails app. /vagrant folder in linux is now mapped to our Windows filesystem. Let’s switch to this folder in linux and create an app there:

cd /vagrant
mkdir rails
cd rails
rails new devhq
cd devhq

We created /vagrant/rails directory, then created app called devhq in this directory and switched to this directory.

7. Now it’s a good time to run newly created app inside of a linux box and open it in a browser on Windows. Type “rails server” and go to “http://localhost:3000&#8221;. Oops! Doesn’t work. The reason is… Rails 4.2. It listens on localhost:3000 by default instead of 0.0.0.0:3000 as before. So you have to stop the server (Ctrl+C) and run “rails s -b 0.0.0.0”. Now you can open your app in the browser.

It’s done! Now you can edit your app with your favorite editor on Windows and run your app on Linux. Cool, isn’t it? Here is the list of commands you should also know:

vagrant ssh – you know already, use this command to log in to your linux box

vagrant up – run the virtual machine

vagrant halt – stop the virtual machine

vagrant reload – reload vagrant configuration, remap file system, remap ports, etc.. Can be useful if you have any errors.

vagrant provision – run provisioning scripts again

sudo reboot – try to reboot linux machine if you have any errors

su – switch to root user. The password is “vagrant”

Knockout Binding for Date Formatting

Recently I’ve been doing a lot of work with Knockout. I’ve run into some scenarios where my screens and user interfaces are needing to be “richer” and I’ve not got the need for Flash or Silverlight. So I’ve been using and learning Knockout. For those not familiar, Knockout is a Javascript library that lets you build dynamic screens and helps you make use of the MVVM pattern. Follow this link if you’d like to read more.

When working with Knockout you use the HTML5 attribute data-bind. Take the following code for example that puts a user’s name in the

This code basically places “Sample User” inside of the span tag. Now let’s say that you would like to put a “date” in there. Maybe your ViewModel included a Date of Birth.

With Knockout, the dateString binding does not exist. However, like any great framework, you can extend behavior. What we need to do is create a Custom Binding. The following code will create our DateString formatter and it leverages the Moment.js library for date formatting. Check it out if you haven’t at this point.

Just a quick basic, from the knockout documentation, “update” will be called once when the binding is first applied to an element, and again whenever the associated observable changes value. Update the DOM element based on the supplied values here. Now that’s out of the way, let’s get to the meat of our function.

The function takes the value of the object as well as any binding options. The value is unwrapped via the ko.utils function which returns the actual object value. The datePattern is fished out of the allBindingsAccessor to build the format that we would like to apply. You can of course customize the “default” behavior here in this method leaving datePattern unneeded in many occasions. Lastly, if the value is defined, I build a date from Moment in the ISO 8601 format and then output the date in the format represented in datePattern.

Pull all of this together, and you get nice and clean date formatting leveraging what Knockout gives you plus adding a little something extra.