Setup script to get Ruby and Rails running on Ubuntu with one command using RailsReady

Posted by Bhushan Ahire | Posted in git, Rails, ruby, Subversion | Posted on 05-12-2011

0

How would you like to get a full Ruby on Rails stack up on Ubuntu with one command?

Now you can by running Rails Ready. Rails Ready is a setup script that gets Ruby and Rails running on a fresh install of Ubuntu with one command (Tested on Ubuntu server 10.04 LTS (Long-term Support)).

This is a brand new project by Josh Frye that he uses all the time to setup VMs, but there’s always testing to be done and improvements to be made.

Running the Script

Check out railsready.sh to see everything Rails Ready is doing.

  sudo wget --no-check-certificate https://github.com/joshfng/railsready/raw/master/railsready.sh && bash railsready.sh

The script will then ask if you want to build Ruby from source or install RVM. If you want to watch the magic happen just run tail -f ~/railsready/install.log.

What gets installed?

  • An updated system (Linux only)
  • Homebrew (OSX only)
  • Ruby 1.9.3 latest patch level (installed to /usr/local/bin/ruby) or RVM running 1.9.3 latest patch level
  • Imagemagick
  • libs needed to run Rails (sqlite, mysql, etc)
  • Bundler, Passenger, and Rails gems
  • Git

All you need to do is install NGINX or Apache, run passenger-install-nginx-module or passenger-install-apache-module, upload your app, point your vhost config to your apps public directory and go!

A note about RVM+passenger+nginx: Passenger installed via RVM can’t locate the OpenSSL package installed on Ubuntu. A user contributed fix is as follows:

rvm remove 1.9.2
rvm package install openssl
rvm install 1.9.2 --with-openssl-dir=$HOME/.rvm/usr
rvmsudo passenger-install-nginx-module

Hope this guide will be helpful to you.

Windows Firewall and non-secure FTP traffic Rule for firewall

Posted by Bhushan G Ahire | Posted in Amazon EC2 | Posted on 13-05-2010

0

Windows firewall can be configured from command line using netsh command. 2 simple steps are required to setup Windows Firewall to allow non-secure FTP traffic

1) Open port 21 on the firewall

netsh advfirewall firewall add rule name=”FTP (no SSL)” action=allow protocol=TCP dir=in localport=21

2) Activate firewall application filter for FTP (aka Stateful FTP) that will dynamically open ports for data connections

netsh advfirewall set global StatefulFtp enable

Hope this information is helpful for you….

How Do I Enable remote access to PostgreSQL database server?

Posted by Bhushan G Ahire | Posted in Amazon EC2, Security | Posted on 11-05-2010

0

By default, PostgreSQL database server remote access disabled for security reasons. However, some time you need to provide the remote access to database server from home computer or from web server.

Step # 1: Login over ssh if server is outside your IDC

Login over ssh to remote PostgreSQL database server:

$ ssh user@remote.pgsql.server.com

Step # 2: Enable client authentication

Once connected, you need edit the PostgreSQL configuration file, edit the PostgreSQL configuration file /var/lib/pgsql/data/pg_hba.conf (or /etc/postgresql/8.2/main/pg_hba.conf for latest 8.2 version) using a text editor such as vi.

Login as postgres user using su / sudo command, enter:

$ su - postgres

Edit the file:

$ vi /var/lib/pgsql/data/pg_hba.conf

OR

$ vi /etc/postgresql/8.2/main/pg_hba.conf

Append the following configuration lines to give access to 10.10.29.0/24 network:

host all all 10.10.29.0/24 trust

Save and close the file. Make sure you replace 10.10.29.0/24 with actual network IP address range of the clients system in your own network.

Step # 2: Enable networking for PostgreSQL

You need to enable TCP / IP networking. Use either step #3 or #3a as per your PostgreSQL database server version.

Step # 3: Allow TCP/IP socket

If you are using PostgreSQL version 8.x or newer use the following instructions or skip to Step # 3a for older version (7.x or older).

You need to open PostgreSQL configuration file /var/lib/pgsql/data/postgresql.conf or /etc/postgresql/8.2/main/postgresql.conf.

# vi /etc/postgresql/8.2/main/postgresql.conf

OR

# vi /var/lib/pgsql/data/postgresql.conf

Find configuration line that read as follows:

listen_addresses='localhost'

Next set IP address(es) to listen on; you can use comma-separated list of addresses; defaults to ‘localhost’, and ‘*’ is all ip address:

listen_addresses='*'

Or just bind to 202.54.1.2 and 202.54.1.3 IP address

listen_addresses='202.54.1.2 202.54.1.3'

Save and close the file. Skip to step # 4.

Step #3a – Information for old version 7.x or older

Following configuration only required for PostgreSQL version 7.x or older. Open config file, enter:

# vi /var/lib/pgsql/data/postgresql.conf

Bind and open TCP/IP port by setting tcpip_socket to true. Set / modify tcpip_socket to true:

tcpip_socket = true

Save and close the file.

Step # 4: Restart PostgreSQL Server

Type the following command:

# /etc/init.d/postgresql restart

Step # 5: Iptables firewall rules

Make sure iptables is not blocking communication, open port 5432 (append rules to your iptables scripts or file /etc/sysconfig/iptables):

iptables -A INPUT -p tcp -s 0/0 --sport 1024:65535 -d 20.04.23.22  --dport 5432 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -s 20.04.23.22 --sport 5432 -d 0/0 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

Restart firewall:

# /etc/init.d/iptables restart

Step # 6: Test your setup

Use psql command from client system. Connect to remote server using IP address 20.04.23.22 and login using postgres username and testdatabase database, enter:

$ psql -h 20.04.23.22 -U postgres -d testdatabase

Setup Capistrano to deploy Rails application on Amazon EC2 with Git

Posted by Bhushan G Ahire | Posted in capistrano, git, Rails, ruby, Security, Subversion | Posted on 17-02-2010

0

1: Create a new Rails app – we’ll call is ‘deploytest’

$ rails deploytest
$ cd deploytest

2: Create a local Git repository for it

$ git init
$ git add *
$ git commit -a -m 'initial commit'
$ git status

3: Create a couple of Capistrano files

$ capify .

4: Edit config/deploy.rb

# The name of your app
set :application, "deploytest"
# The directory on the EC2 node that will be deployed to
set :deploy_to, "/var/www/apps/#{application}"
# The type of Source Code Management system you are using
set :scm, :git
# The location of the LOCAL repository relative to the current app
set :repository,  "."
# The way in which files will be transferred from repository to remote host
# If you were using a hosted github repository this would be slightly different
set :deploy_via, :copy

# The address of the remote host on EC2 (the Public DNS address)
set :location, "ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com"
# setup some Capistrano roles
role :app, location
role :web, location
role :db,  location, :primary => true

# Set up SSH so it can connect to the EC2 node - assumes your SSH key is in ~/.ssh/id_rsa
set :user, "root"
ssh_options[:keys] = [File.join(ENV["HOME"], ".ssh", "id_rsa")]

The only account on a default EC2 instance is root. You probably want to create a second user that is responsible for your application.

5: Copy your SSH public key to your EC2 node

$ scp -i ~/my-ec2-keypair ~/.ssh/id_rsa.pub root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:/root/.ssh/authorized_keys2

NOTE the filename authorized_keys2 – not authorized_keys!!

6: Setup the EC2 node for Capistrano deployment.
From your LOCAL machine, not the EC2 node:

$ cap deploy:setup

7: Finally, deploy your application

$ cap deploy

You will see lots of output and with this dummy application some of those will report errors/warnings. Don’t worry about that for now.

8: Check that the Deployment was successful
Connect to the EC2 node with SSH the regular way, cd to the app directory and check that everything is there. If that is all working then you are ready to deploy a real application and add custom tasks for managing the database, restarting the server etc.

Bear in mind that Capistrano add new ‘releases’ of your software in separate directories and symlinks the ‘current’ directory to the latest. So the root of your deployed application is the ‘current’ subdirectory.

Hope this will help you setting up your ec2 instance with capistrano.

Step-By-Step setup slicehost server (fedora) for rails application.

Posted by Bhushan Ahire | Posted in capistrano, Rails, Subversion | Posted on 22-07-2009

1

Install and Configure FTP Server in Amazon EC2 instance

Posted by Bhushan Ahire | Posted in Amazon EC2, Security | Posted on 15-04-2009

0

For many users, running FTP Sever in Amazon EC2 instance is headache at the first time. You need to experiment before being able to transfer data. The main problems are Ingress firewall in Amazon environment and NAT traversal.

Here I’m using vsftp (vsfptd) Server, which is one of the most popular and easy to configure. The instance is running from base Fedora 4 AMI but the setup should be identical to other Red Hat based distros.

Install vsftpd FTP server, if not installed earlier:

# yum install vsfptd

Its upto you which FTP method i.e. Active or Passive you want to use. The problem with active mode is that your computer is sending a request out of port 21 when all of a sudden, the server attempts to initiate a request with your computer on port 20. Since communication on port 21 does not imply communication on port 20, it appears as if some unauthorized host has attempted to initiate a new connection with your computer. Kind of sounds like a hack right? Your firewall may think so too (or your NAT router may have no idea to which computer to route the request). Active mode is not used as default method of ftp transfer in many clients these days.

On the other hand, as the Ingress firewall is running in AWS, from the firewall’s standpoint, to support passive mode FTP the following communication channels need to be opened:

FTP server’s port 21 from anywhere (Client initiates connection).
FTP server’s port 21 to ports > 1023 (Server responds to client’s control port).
FTP server’s ports > 1023 from anywhere (Client initiates data connection to random port specified by server).
FTP server’s ports > 1023 to remote ports > 1023 (Server sends ACKs (and data) to client’s data port).

That second part is the problem: FTP server listens on a random port and hands that back to the client, so the client initiates a connection to a random server port, which you must allow.

Opening up all ports > 1023 isn’t so good for security. But what you can do is allow the ports through the distributed firewall and then setup your own filtering inside your instance. Instead, you would better open a fixed number of ports (such as 1024 to 1048) and configure your FTP Server to only use that ports.

Check whether required ports are open or not in your EC2 security group. (if you are unaware about security group, it should be ‘defaul’ unless you created a new one).

# ec2-describe-group

This command will print all ports which are currently open. If you dont find port 20,21,1024-1048 then you need to open these ports but if you dont find the command itself i.e.
# ec2-describe-group
-bash: ec2-describe-group: command not found

You need to install ec2 command line tools. You can find them here and the instructions to setup/configure can be found here.

Open the ports now:

# ec2-authorize default -p 20-21
# ec2-authorize default -p 1024-1048

Here, ‘default’ is the name of security group. You can also open ports for specific IPs. For ease of use, you better install ElasticFox, a firefox extension to manage EC2 stuff. you can find more about it here.

At this moment, you can start your FTP server and if you try to connect it, the process will get failed. By checking logs, you should find something like:

Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is current directory.
Command: TYPE A
Response: 200 Type set to A
Command: PASV
Response: 227 Entering Passive Mode (216,182,238,73,129,75).
Command: LIST
Error: Transfer channel can't be opened. Reason: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Error: Could not retrieve directory listing

Time to configure vsftpd.conf file:
# vi /etc/vsftpd/vsftpd.conf
---Add following lines at the end of file---
pasv_enable=YES
pasv_min_port=1024
pasv_max_port=1048
pasv_address=Public IP of your instance

Put public IP of your EC2 instance and then Save the file. Now restart the server:

# /etc/init.d/vsftpd restart

Setting up a new remote git repository

Posted by Bhushan Ahire | Posted in git, Rails | Posted on 19-02-2009

2

For the impatient

Set up the new bare repo on the server:

$ ssh myserver.com
Welcome to myserver.com!
$ mkdir /var/git/myapp.git && cd /var/git/myapp.git
$ git --bare init
Initialized empty Git repository in /var/git/myapp.git
$ exit
Bye!

Add the remote repository and push:

$ cd ~/Sites/myapp
$ git remote add origin ssh://myserver.com/var/git/myapp.git
$ git push origin master

Set the local master branch to track the remote branch.

Read further for a step-by-step explanation of what’s going on.

Pre-flight sanity check

Setting up a remote repository is fairly simple but somewhat confusing at first. Firstly, let’s check out what remote repositories are being tracked in our git repo:

$ cd ~/Sites/myapp
$ git remote

None. Looking good. Now let’s list all the branches:

$ git branch -a
* master

Just one branch, the master branch. Let’s have a look at .git/config:

$ cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true

A pretty bare-minimum config file.

Creating the bare remote repository

Before we can push our local master branch to a remote repository we need to create the remote repository. To do this we’ll ssh in and create it on the server:

$ ssh myserver.com
Welcome to myserver.com!
$ cd /var/git
$ mkdir myapp.git
$ cd myapp.git
$ git --bare init
Initialized empty Git repository in /var/git/myapp.git
$ exit
Bye!

A short aside about what git means by bare: A default git repository assumes that you’ll be using it as your working directory, so git stores the actual bare repository files in a .git directory alongside all the project files. Remote repositories don’t need copies of the files on the filesystem unlike working copies, all they need are the deltas and binary what-nots of the repository itself. This is what “bare” means to git. Just the repository itself.

Adding the remote repository to our local git repository configuration

Now that we’ve created the remote repository we’ll add it to our local repository as a remote server called “origin” using git remote add, which is just a nicer way of updating our config file for us:

$ git remote add origin ssh://myserver.com/var/git/myapp.git

Let’s see what it added to the config file:

[core]
  repositoryformatversion = 0
  filemode = true
  bare = false
  logallrefupdates = true
[remote "origin"]
  url = ssh://myserver.com/var/git/myapp.git
  fetch = +refs/heads/*:refs/remotes/origin/*

We now have a remote repository “origin” that will fetch all of it’s refs/heads/* branches and store them in our local repo in refs/remotes/origin/* when a git fetch is performed.

Pushing to the remote repository

The time has come to push our local master branch to the origin’s master branch. We do that using the git push command.

$ git push origin master
updating 'refs/heads/master'
  from 0000000000000000000000000000000000000000
  to   b379203bc187c2926f44a71eca3f901321ea42c6
 Also local refs/remotes/origin/master
Generating pack...
Done counting 1374 objects.
Deltifying 1374 objects...
 100% (1374/1374) done
Writing 1374 objects...
 100% (1374/1374) done
Total 1374 (delta 89), reused 0 (delta 0)
refs/heads/master: 0000000000000000000000000000000000000000 -> b379203bc187c2926f44a71eca3f901321ea42c6

and that’s all, folks. Further pushes can be done by repeating the git push command.

Now you can tell your co-conspirators to:

$ git clone ssh://myserver.com/var/git/myapp.git

and push and pull to your heart’s content.

Track the remote branch

You can specify the default remote repository for pushing and pulling using git-branch’s track option. You’d normally do this by specifying the --track option when creating your local master branch, but as it already exists we’ll just update the config manually like so:

[branch "master"]
  remote = origin
  merge = refs/heads/master

Now you can simply git push and git pull.

Sharing the remote repository with the world

If you want to set it up as a public repository be sure to check out the Git manual’s chapter on public git repositories.

Working with remote repository branches

git remote show is used to inspect a remote repository. It goes and checks the remote repository to see what branches have been added and removed since the last git fetch.

Doing a git remote show at the moment only shows us the remote repo’s master branch which we pushed earlier:

$ git remote show origin
* remote origin
  URL: ssh://myserver.com/var/git/myapp.git
  Tracked remote branches
    master

Let’s create a new local git repository and push to a new branch on the remote repository. We can then use git remote show to see the new remote branch, git fetch to mirror it into our local repo and git checkout --track -b to create a local branch to do some work on it.

We’ll start by creating a new local repo and pushing some code to a new branch in the remote repository.

$ mkdir /tmp/other-git
$ cd /tmp/other-git
$ git init
Initialized empty Git repository in /tmp/other-git
$ git remote add origin ssh://myserver.com/var/git/myapp.git
$ echo "Rails 2... woo" > afile
$ git add afile
$ git commit -m "Added afile" 
Created initial commit 0ac9a74: Added afile
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 something
$ git push origin master:refs/heads/dev/rails-2
updating 'refs/heads/rails-2' using 'refs/heads/master'
  from 0000000000000000000000000000000000000000
  to   0ac9a7457f4b21c9e058d4c54d262584bf35e528
 Also local refs/remotes/origin/rails-2
Generating pack...
Done counting 3 objects.
Deltifying 3 objects...
 100% (3/3) done
Writing 3 objects...
 100% (3/3) done
Total 3 (delta 0), reused 0 (delta 0)
Unpacking 3 objects...
 100% (3/3) done
refs/heads/rails-2: 0000000000000000000000000000000000000000 -> 0ac9a7457f4b21c9e058d4c54d262584bf35e528

Now let’s switch back to our old git repository and see if it detects the new branch on the remote repository:

$ git remote show origin
* remote origin
  URL: ssh://myserver.com/var/git/myapp.git
  New remote branches (next fetch will store in remotes/origin)
    dev/rails-2
  Tracked remote branches
    master

Let’s update our mirror of the remote repository by doing a git fetch:

$ git fetch
* refs/remotes/origin/master: storing branch 'dev/rails-2' of ssh://myserver.com/var/git/myapp.git
  commit: b379203
$ git remote show origin
* remote origin
  URL: ssh://myserver.com/var/git/myapp.git
  Tracked remote branches
    master
    dev/rails-2

We should now be able to see this in a our list of remote branches:

$ git branch -a
* master
origin/dev/rails-2
origin/master

If we then wanted to do some work on this remote dev/rails-2 branch we create a new local tracking branch like so:

$ git checkout --track -b dev/rails-2 origin/dev/rails-2
Branch dev/rails-2 set up to track remote branch refs/remotes/origin/dev/rails-2.
Switched to a new branch "dev/rails-2"

To keep up-to-date and push new changesets we simply use git push and git pull when working in the local dev/rails-2 branch.

Also notice, like we manually changed for master, .git/config has a new entry for this new tracking branch:

[branch "dev/rails-2"]
  remote = origin
  merge = refs/heads/dev/rails-2

god Server process monitoring and notifying tool written in ruby

Posted by Bhushan Ahire | Posted in Rails, ruby | Posted on 18-03-2008

0

Progress on god is moving along as quick as ever. Most interestingly you’ll find several useful new command line functions:

  • god status prints out the status of each Watch
  • god log shows realtime logs for a specific Watch (even if you don’t have god logging to file)
  • god load loads or reloads a config file into a running god instance
  • god terminate stops all Watches and then stops god (useful when testing your setup)

The logging system has been beefed up with proper timestamps and criticality levels. Log messages are more complete overall. You can also get the STDOUT/STDERR of a god-daemonized process written to a log file by specify ‘w.log = ‘ in your Watch config.

If you let god daemonize your process for you, there’s no need to provide a stop command. A default killing lambda will take care of gracefully (or not so gracefully if necessary) stopping your god-daemonized process.

The validity of your config file is checked better than previous versions to point you to the problem area of your config.

The bug that prevented group control from working has been fixed so you can now start/stop/etc groups of Watches.

Updated documentation is now available on the website:

http://god.rubyforge.org/

WHAT IS GOD?

God is an easy to configure, easy to extend monitoring framework written in Ruby.

Keeping your server processes and tasks running should be a simple part of your deployment process. God aims to be the simplest, most powerful monitoring application available.

INSTALL

sudo gem install god

FEATURES

  • Config file is written in Ruby
  • Easily write your own custom conditions in Ruby
  • Supports both poll and event based conditions
  • Different poll conditions can have different intervals
  • Easily control non-daemonized processes

EXAMPLE

The easiest way to understand how god will make your life better is by looking at a sample config file. The following configuration file is to keep the mongrels running:

# file:      application_name.god
# run with:  god -c /path/to/application_name.god
# 
# This is the actual config file used to keep the mongrels of
# application.com running.
RAILS_ROOT = "/var/www/application_name/current"
%w{8001 8002 8003}.each do |port|
  God.watch do |w|
    w.name = "application_name-mongrel-#{port}"
    w.interval = 30.seconds # default
    w.start = "mongrel_rails cluster::start --only #{port} 
      -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
    w.stop = "mongrel_rails cluster::stop --only #{port} 
      -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
    w.grace = 10.seconds
    w.pid_file = File.join(RAILS_ROOT, "log/mongrel.#{port}.pid")    
    w.behavior(:clean_pid_file)
    w.start_if do |start|
      start.condition(:process_running) do |c|
        c.interval = 5.seconds
        c.running = false
      end
    end 
    w.restart_if do |restart|
      restart.condition(:memory_usage) do |c|
        c.above = 150.megabytes
        c.times = [3, 5] # 3 out of 5 intervals
      end    
      restart.condition(:cpu_usage) do |c|
        c.above = 50.percent
        c.times = 5
      end
    end
  end
end

Configuration for nginx and mysql
I found a gr8 post for sample configuration here

DOCS

Detailed documentation is available at http://god.rubyforge.org/

Original Source…

Shorcuts for kill and restart rails server

Posted by Bhushan Ahire | Posted in Rails | Posted on 22-01-2008

0

More lovely alias commands… this time to kill/restart Rail’s script/server from any Terminal session or login on your box… (as long as your the same user).

alias dierails='ps -a|grep "/usr/local/bin/ruby script/server"|grep -v "grep /usr"|cut -d " " -f1|xargs -n 1 kill -KILL $1'
alias resetrails='ps -a|grep "/usr/local/bin/ruby script/server"|grep -v "grep /usr"|cut -d " " -f1|xargs -n 1 kill -HUP $1'