Deploying multiple applications on single server using nginx and unicorn

This summary is not available. Please click here to view the post.

Setting up Sunspot-Solr in Production with jdk And Apache Tomcat

As you all know Apache Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, rich document (e.g., Word, PDF) handling, and geospatial search. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of many of the world's largest internet sites.

Sunspot is a ruby library for the expressive, powerful interaction with the solr search engine. Sunspot is built on the top of RSolr library. Which provides a low level interface for solr interaction, Sunspot provides a simple, intuitive, expressive DSL backed by powerful features for indexing objects and searching for them.

When we are working with RoR we can install sunspot by adding two gems to the Gemfile and do bundle install.

gem 'sunspot_rails'

gem 'sunspot_solr'

Then do bundle insatall.

bundle install

Then generate a default configaration file.

rails generate sunspot_rails:install

Now lets look how we can setting up solar on the system. For installing Solr we need Java in our system. So first we can set up  and configure Java. Install latest jdk to your system. You can either go to sun website and download the package and install it or use apt-get. For the later first run

apt-get update

It will update our source and find the latest packages. Then run

apt-get install openjdk-version-jdk

After completion of installation you should set the path and the java_home to .bashrc file for both root and local.

1. /home/user/.bashrc
2. /root/.bashrc
Then add
1. export PATH=~/java_installation_path(eg:/usr/lib/jvm/java-version)/bin:$PATH
2. export JAVA_HOME=~/java_installation_path(eg:/usr/lib/jvm/java-version)

After adding the above please do "source .bashrc" to affect changes.

If the above mentioned didn't went well please go through the below link. It is a lengthy process bu the job will get done definitely.

http://www.wikihow.com/Install-Oracle-Java-on-Ubuntu-Linux

Next step is installing latest tomcat to your system.

apt-get install tomcat7 tomcat7-admin tomcat7-common tomcat7-user tomcat7-docs

apt-get install libmysql-java(optional)

To test tomcat run

/etc/init.d/tomcat7 start

/etc/init.d/tomcat7 status

Now we need to give permission to user for accessing this.

nano /etc/tomcat7/tomcat-users.xml

It will open the above file and add the following lines to the file.

role rolename="manager"
role rolename="admin"
user username="admin" password="SPECIAL" roles="manager,admin"

Note: Each line should add as xml, means there should be starting and ending tags.

Then add Data Directory and give permission.

mkdir /var/lib/tomcat7/solr/data
chown -R tomcat7:tomcat7 /var/lib/tomcat6/solr/data/

There are some more permissions we have to give.

chown -R tomcat7:tomcat7 /var/lib/tomcat7/
chmod 775 /var/lib/tomcat7/conf/tomcat-users.xml
chmod 775 /var/lib/tomcat7/conf/Catalina/localhost/solr.xml

Wow we almost done. It is the time to test. Re-start Tomcat!!!!

/etc/init.d/tomcat7 restart
To test:
curl http://127.0.0.1:8080/solr/admin/

One last thing!!!!!!!! If you are using Rails for your site please do the following step. Copying the rails config file:

cp RAILS_ROOT/solr/conf/schema.xml /var/lib/tomcat7/solr/conf/schema.xml
/etc/init.d/tomcat7 restart

An Introduction to Sql Procedures

As I mentioned in my previous post "Lets play with Sql" we can use SQL  programmatically for data manipulation. SQL Procedures and functions helps to full fill this. Using procedures we can execute number of operations or queries with in one procedure call. More over we can call a procedure with in another procedure like other languages.

Syntax of Creating Procedure is

CREATE PROCEDURE procedure_name()

          BEGIN
            .
            .
            .
          END 

With in BEGIN and END we write the procedure body includes queries,loops,string manipulations etc. We can store this procedure for future use. We can call the stored procedure in future. The syntax for calling procedure is

CALL procedure_name()

There is another good way to create and call procedures. We can store procedures in a text file. Then first load this text file to sql by this time the procedure will store. For this we can use

source file_name

in MySQL prompt. After loading we can call the procedure using CALL command.

We can Drop an existing procedure using command
DROP PROCEDURE procedure_name

Now we can go more deeper. For that Lets look in to one example.

DELIMITER $$
 DROP PROCEDURE IF EXISTS WhileLoopProc$$
 CREATE PROCEDURE WhileLoopProc()
 BEGIN
   DECLARE x  INT;
   DECLARE str  VARCHAR(255);
   SET x = 1;
   WHILE x  <= 30 DO
     SET str = (SELECT name FROM sanity WHERE id=x);
     SET  x = x + 1;
     SELECT str;
   END WHILE;
 END$$
DELIMITER;

In this example we have already discussed the section before 'BEGIN'. DECLARE is the key word using for declaring variables and 'SET' is using for assigning value to a variable.The next section is a while loop for selecting name corresponds to id and print the same. For printing a value we use 'SELECT' key word.

If you understand above example it is time to go little more deeper. Before that it is better to go through the link given below.
http://dev.mysql.com/doc/refman/5.0/en/string-functions.html
It gives descriptions about various string functions.

DELIMITER $$
 DROP PROCEDURE IF EXISTS CursorProc$$
 CREATE PROCEDURE CursorProc()
 BEGIN
  DECLARE str  VARCHAR(255);
  DECLARE cur1  CURSOR FOR SELECT name FROM sanity LIMIT 500;
  OPEN cur1;
  read_loop: LOOP
   FETCH cur1 INTO str;
   SET str = TRIM(LEADING '"' FROM str);
   SET str =TRIM(TRAILING '"' FROM str);
   CALL selct_proc( str);
  END LOOP; 
  CLOSE cur1;
 END$$
DELIMITER ;

DELIMITER $$
DROP PROCEDURE IF EXISTS selct_proc$$
CREATE PROCEDURE selct_proc(INOUT str1 VARCHAR(25))
BEGIN
 DECLARE str_dot  VARCHAR(255);
 DECLARE str_rplce  VARCHAR(255);
 DECLARE first_str  VARCHAR(255);
 SET str_dot = LOCATE(".",str1);
 IF str_dot != 0 then
  SET str_rplce = REPLACE(str1, '.', ' ');
  SET first_str = SUBSTRING_INDEX(str_rplce,' ',1);
  SELECT str_rplce,first_str;
 END IF;
END$$
DELIMITER ; 

This above example covers almost all basics like Loop, If conditions calling a procedure with in another procedure,passing parameters and using strong string functions for manipulating strings.


In the above example we using a variable cur1 it is for storing all the data after executing the query. Then inside a loop using FETCH key word we are accessing each value into a string. Then using some string function we styling the string. Then we are calling a procedure inside and passing a variable. In the second one we accessing the variable,here you see a key word INOUT this fore returning the value back. Here we are using a small IF construct to familiar with that.

This is only a small introduction to the topic. But by going through this you can start to play with procedures. And the main advantage of using procedures is its speed of execution. It is very much faster than writing script  for job done. So lets try.

Setting up Ruby and Rails using RVM in ubuntu systems

As a RoR developer the first hurdle I faced was proper installation of rails using rvm, same as many others. Installed and re-installed number of times, followed number of tutorials. From that experience trying to consolidate the steps need for installation that I found from different tutorials.

 Lets directly go to the steps.

First we need to set up some platform for installing the RVM and Rails.

sudo apt-get install build-essential git-core curl libmysqlclient16-dev nodejs

these are some plug-ins to make the job easy.

To install the latest RVM first we need to remove the old version if it is there

sudo apt-get --purge remove ruby-rvm 

Then we need to remove two files from the system

sudo rm -rf /usr/share/ruby-rvm /etc/rvmrc /etc/profile.d/rvm.sh 

We can confirm the removal using the below command

env | grep rvm 

If the RVM is still there it shows the path as out put.

The next step is downloading and installing latest stable RVM available. For that we can use curl command.

curl -L get.rvm.io | bash -s stable --auto 

The above command will download and install the latest stable RVM package. At the end of the installation the system may throw some messages by asking some plug-ins are missing. Mostly the the system will show the required plug-in name. Then we can install those using apt-get command.

Otherwise from my experience you can give the below command to solve the problem. This will install almost all dependencies needed by RVM.

sudo apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion

Then run the following command for confirmation.

source "/usr/local/rvm/scripts/rvm" 
or
source ~/.rvm/scripts/rvm

We are doing this to make RVM to use as function. Then run

type rvm | head -n 1

if you are getting  rvm is a function as out-put the problem is solve.
Now we done with all the foundations. Next step is installing ruby. We can install different versions using RVM.

rvm install ruby-version(eg: 1.9.2 or 1.9.3 ) 

We only need to give the version number, no need of giving prefix ruby 


If number of versions are there in our system then we can use the following command to use a desired version as default.

rvm --default use ruby-version(eg: 1.9.2 or 1.9.3) 

Then we can check which ruby we are using as default by the following command.

ruby -v

Now we can install rails to our system. We will install rails using "gem" installer. If the system throwing any error like gem is not found,you need to install the gem installer first. After that we can install rails.

gem install rails

If you use the above command it will install the latest version of rails. If spesific version need to be installed,can use the below command

gem install rails 'version'(eg: 3.1.2 or 3.1.3)

Run the below command in your terminal

[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"

If you want to disable installation of gem documentation while installing new gems,you can try the following command. It is optional.

echo "gem: --no-rdoc --no-ri" > ~/.gemrc 

The above command will add this line to "gemrc" file.

Lets enjoy the rails experience.

Lets Play With SQL

In My knowledge SQL or MySQL (Structured Query Language) is a tool for storing data in form of rows and columns. But when I started working deeply with SQL I realized that it is not only a query language ,we can use it as a power full programming tool for data manipulation.

We all know about basic SQL commands like ‘Create Table’,'Insert Into’, ‘Select’, ‘Delete’,'Drop’ etc. Here I am try to introduce some commands that not regularly use, but very use full.

Usually LIKE is using for pattern matching. But we can use ‘LIKE’ also for creating a new table structure as an existing table.

"CREATE TABLE new_table_name LIKE existing_table"

After creating table structure we can copy contents from other table using INSERT INTO command

"INSERT INTO new_table SELECT * FROM existing_table"

It copy all data from one table to another. If we need to copy only specific column we can specify column names

"INSERT INTO new_table (col1,col2,......)SELECT col1,col2,.... FROM existing_table"

we can also give specify copy data with in a limit.

"INSERT INTO new_table SELECT * FROM existing_table LIMIT 1000 OFFSET 1000"

It copies data from 1000th raw up to 2000th row.

We can rename a table using rename command;

"RENAME TABLE tanle_name TO new_name"

There may be some situations that we need to back up all the data from a table or back up entire data base. For this purpose we use MySQL dump command. These are very use full system commands for creating back up.

MySQL Dump Export command for backing up:

Dump Database:

"mysqldump --user='username' --password='password' db_name > filename.sql"

Only Database Structure:

"mysqldump --user='username' --password='password' --no-data db_name > filename.sql"

Database Data:

"mysqldump --user='username' --password='password' --no-create-info db_name > filename.sql"

Dump Severel Databases in to a single file:

"mysqldump --user='username' --password='password' --databases  db_name1,db_name2 > filename.sql"

Dump All databases in Sever:

"mysqldump --user='username' --password='password' --all-databases  > filename.sql"

Restore and Import Databases:

"mysql --user='username' --password='password'  db_name < filename.sql"

To Export A Table Structure:

"mysqldump --user='username' --password='password' db_name table_name >filename.sql"

To Export A Table data:

"mysqldump --no-create-info --compact --user='username' --password='password' db_name table_name > table_name.sql"

Import Table:

"mysql --user='username' --password='password'  db_name < table_name.sql"

Power Of Walk

Os.path.walk is a fruitful system command in python. Using walk we can easily parse recursively through the folder.It gives the file names with in the folder and in the sub folders.  The syntax of walk is very simple. os.path.walk takes exactly three arguments. Syntax of os.path. walk is-

os.path.walk(parent_folder,function_call,pattern)

The first argument ‘parent_folder’ is the directory name that we need to parse. function_call is a function call that used to parse each folder and sub folder. It also takes three arguments. The third one pattern is actually a filtering parameter. We can give the extensions of file that we need to manipulate. If we need to get only mp3 files then we can give ‘*.mp3′ for pattern. Like we can give any extension. For getting all the files we can give ‘*.*’ for pattern.

Here am giving a small example to show the power of walk.

import sys
import shutil
import re
from fnmatch import fnmatch
import os, os.path

global destntn_dir
destntn_dir = None

def mp3_finder(pattern, dir, files):
 for filename in files:
  if fnmatch(filename, pattern):
   file_path =  os.path.join(dir, filename)
   #Copying files to destination folder
   shutil.copy(file_path,destntn_dir)

def main():

 ## Collect  command line arguments
 parent_folder = sys.argv[1]
 global destntn_dir
 destntn_dir = sys.argv[2]
 #Creating Destination Folder
 os.mkdir(destntn_dir)
 ## Walk folder
 os.path.walk(parent_folder,mp3_finder, '*.mp3')

if __name__=="__main__":
  main()

The above example copy all the mp3 files from the given folder and its sub folders to another folder. Here we are creating the destination folder inside the program.

Usage:
Run the program in the console using python command. We can give the source folder and destination folder as command line arguments. Please give the folder names along with their paths. Here the destination folder will create according to the path.