Logo

Who is talking?

Archive

Inside Git Guts

over 3 years ago | Shadab Ahmed: Shadab's Blog

TLDR; Git is greatly extensible. I made many custom commands to create an interesting talk on "The Insides of Git". Skip the text and just watch the video for details. Back in June last year in the cool city of Pune at the RubyConf India 2013, I gave a talk on Git. It was aptly titled Inside Git Guts with Ruby. The idea was to explore what happens inside the .git folder when you do git operations like commit, push, merge etc. The talk was far more about git than ruby. Infact, the part with Ruby in the title was added just to ensure a better chance of my proposal getting selected for RubyConf. Although, all of the tools I showed in the talk were written in Ruby, the actual ruby code shown was just about 4 lines. Now, here's the actual talk: The talk was greatly inspired by the movie Inception. Infact I even had a command git inception which gave the following output(truncated): $ git inception .git .git .git .git .git .git .git .git <--- 2nd Level Gitception .git The Inception bit was actually the most interesting part of the talk. There were other super cool commands like - git music, git autocommit, git server, git fireworks, git quote, git about and more Lots of folks have asked me to share the code and the instructions, so here's the code. To get all the tools on your system: gem install git_guts

Bang Bang !!

about 4 years ago | Shadab Ahmed: Shadab's Blog

The more you learn about the linux shell, the more you love it. Reason I am saying this because I recently saw this: sudo !! This made me immediately google it and I found an excellent reference here. This is a quick summary of the source: $ !! # Repeat last command $ !x # Repeat last command that started with x $ !?x # Repeat last command that has the substring x $ !10 # Repeat 10th command in the history file $ !-10 # Repeat 10th from last command in the history file $ !!* # Fetch parameters from last command $ !!^ # Fetch first parameter from last command $ !!$ # Fetch last parameter from last command $ !!3 # Fetch third parameter from last command $ !!:s/foo/bar/ # Repeat last command substituting foo for bar If you append :p to any of the above commands then it will only print the command than running it Infinite Bang Bang Now what if the last command also contains a !!. Would it become recursive ? Let's see: # : is noop operator. This command sets up our base. # Note that the !! inside single quote doesn't expand to the previous command $ : '!!' # Let's start our recursive operation in here. This is the first sequence in the chain # Only the !! inside double quotes expands to the previous command - : '!!' $ : "!!" '!!' : ": '!!'" '!!' # Pressing up arrow, we have a second sequence # Once again, only !! inside double quotes expands and !! appears 3 times after expansion $ : ": '!!'" '!!' : ": ': ": '!!'" '!!''" '!!' # Now, third sequence - !! appears 5 times $ : ": ': ": '!!'" '!!''" '!!' : ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!' # Fourth Sequence - !! appears 17 times $ : ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!' : ": ': ": '!!'" ': ": ': ": ': ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!''" ': ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!'''" ': ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!''''" '!!' # Fifth $ : ": ': ": '!!'" ': ": ': ": ': ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!''" ': ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!'''" ': ": ': ": '!!'" ': ": ': ": '!!'" '!!''" '!!'''" '!!''''" '!!' : ": ': ": '!!'" ': ": ': ": ': ": ': ": '!!'" ': ": ': ": ': ": ': ": '!!'" ': ": ': ": ': ": ' ... # !! appears 161 times In the sixth sequence !! appears 15681 times and in the seventh, more than 150 million times. Do not even try to run it for the 7th iteration unless you want to end up with a 2GB history file and your shell stuck for hours at 100% CPU. The 8th sequence has more than 16 Quadrillion bang bangs Now, I haven't been able to figure out the exact equation yet for the number of !! in nth sequence, but this is what happens: Any !! not surrounded by any quotes is expanded to the previous command !! surrounded only by single quotes '!!' do not expand !! surrounded by double quotes at the outermost level are expanded, even if that bang bang is nested inside single quotes. For e.g. " '!!' " The approximate equation is like this: f(n) ~ f(n-1)*f(n-1)/2 + f(n-1)/2 Let me know if you can figure out the exact equation. The above commands have been tested on sh and bash. Update: Thanks to Volker, we have code to calculate the exact number of bang bangs in nth sequence, here, which works till the 7th sequence. Volker has even submitted the Bang Bang Numbers at OEIS and the sequence has been accepted. View it over here Update2: Robin Houston wrote an analysis of these numbers on his blog. He also wrote this code for generating the numbers in the Bang Bang Numbers series: # /usr/bin/env python p, q = 2, 4 for i in range(4, 16): print "%d: %d" % (i, 4*p + 2*q + 1) p,q = 3*p*p + 2*p + p*q, 2*q + 2*p*p + 2*p*q

Split Tunneling on VPN via Routing Table

about 4 years ago | Shadab Ahmed: Shadab's Blog

Split Tunneling Whenever I connect to VPN on my mac, my default route is modified to this: $ netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default utun1 UCS 34 0 utun1 default 192.168.2.1 UGScI 0 0 en0 What this does is, route all my traffic, even the one which is outside of VPN network routed through the VPN interface utun1. The second default route via 192.168.1.1 is not even used. This is unnecessary and sometimes counterproductive as the VPN network takes extra load on bandwidth/resources for the IPs outside of its network and even bans sites which do not require banning. You can easily change your routing table to circumvent this using the script below. This way you access public IPs directly and private IPs over VPN. This concept is called Split Tunneling Modifying Routing Table Now to modify routing table for split tunneling, all you need to do is to find out the subnet of the IPs in your VPN you need access to. Let't take for example, if the subnets of my VPN private space are 10.109.0.0 and 10.110.0.0 (Any IPs starting with 10.109 and 10.110): #! /usr/bin/env bash if (( EUID != 0 )); then echo "Please, run this command with sudo" 1>&2 exit 1 fi WIRELESS_INTERFACE=en0 TUNNEL_INTERFACE=utun0 GATEWAY=$(netstat -nrf inet | grep default | grep $WIRELESS_INTERFACE | awk '{print $2}') echo "Resetting routes with gateway => $GATEWAY" echo route -n delete default -ifscope $WIRELESS_INTERFACE route -n delete -net default -interface $TUNNEL_INTERFACE route -n add -net default $GATEWAY for subnet in 10.109 10.110 do route -n add -net $subnet -interface $TUNNEL_INTERFACE done Just save the script as /usr/bin/vpn and whenever you connect to vpn, just run sudo vpn. This is what the script does: Finds wireless router IP (en0) Remove the default routes set by VPN Set the IP found for en0 as the default gateway Add the specific routes for your VPN subnet If you are connecting over wired network, just change the en0 in script to en1. Don't change your VPN DNS to make sure you can resolve private domains on the VPN. Looking at the routing table again after running the script: $ netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.2.1 UGSc 1 0 en0 ... 10.109 utun1 USc 0 0 utun1 10.110 utun1 USc 0 0 utun1 As you can see now, our default gate is back to wifi router IP - 192.168.1.1 on en0 and there are some routes for the subnets inside the VPN. You can test the route as well with a route command. Route for any IP outside of your VPN: $ route get google.com route to: lga15s29-in-f5.1e100.net destination: default mask: default gateway: 192.168.1.1 interface: en0 flags: <UP,GATEWAY,DONE,STATIC,PRCLONING> recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 Route for IP inside your VPN: $ route get 10.109.10.135 route to: 10.109.10.135 destination: 156.107.0.0 mask: 255.255.0.0 interface: utun0 flags: <UP,GATEWAY,DONE,STATIC,PRCLONING> recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1280 0 To reset the routing table back, just disconnect from the VPN. Same script can be used on Linux with some modifications.

Bypassing VPN via Routing Table

about 4 years ago | Shadab Ahmed: Shadab's Blog

Whenever I connect to Cisco IPSec VPN on my mac, my default route is modified to this: netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire *default utun1 UCS 34 0 utun1 default 192.168.2.1 UGScI 0 0 en0

Logstasher - For awesome Rails logging

over 4 years ago | Shadab Ahmed: Shadab's Blog

Rails Logs Rails logging is a mess, atleast the default one. I've even seen some gems, spewing garbage into the logfile with no predefined format. Good thing is that you can easily fix/customize it. Infact, you can customize it so much, that you can have this: What's there in these supposedly awesome logs ? Superfast search backend UI is much much cooler than, what the single screenshot shows You can add custom fields such as username and custom rails instrumentations Consolidation of logs across multiple servers Email notifications on events such as exceptions How it's done ? The slick UI with superfast search is made possible with Rails Logs using: Logstasher Gem - A rails gem I wrote. It makes rails, generate logstash compatible logs with minimal overhead Logstash - An open source log collector Kibana - A ruby/rack application for visualization and searching the logstash index Logstasher Gem I have already written much about it on the README at Github. Please check it out to know the usage/installation Logstash Now this is the core of the logging system. It collects and stores the logs. It has a very clean seperation of concerns, basically: Inputs - The source for logs. There are tons of options for input sources. We'll be using file input for local logs and lumberjack for remote logs. Filters - These are tranformations you would like to apply on the input log. Used for sanitizing and appending any extra information. Since logstasher already sends sanitized logs, we'll not need any filters, thus saving on some processing. Ouputs - Again we have many options for outputs. We will use the email and elastic-search output. See the full list of input, filter and output options here Setting up logstash You can compile the jar from source code or download it. My preference is to compile, since it is newer (more fixes). Copy the jar file to /opt/logstash/logstash.jar Set it up as a service - Ubuntu and SUSE SLES init.d scripts Setup your config file - See sample config file. Copy it to /etc/logstash/logstash.conf. The email output functionality is not perfect yet. I am waiting still for this pull request to be merged. Once your logstash is setup, you now need to configure input source. If the rails app is on the same machine, you can skip the lumberjack section. For output we're using the embedded elastic-search, so no need for any extra setups. Lumberjack If your app resides on a remote server, or there are multiple app servers, you need to transport the logs to logstash. Lumberjack is a very fast and small utility to do the same with minimal memory usage. It also encrypts the logs, so super secure as well. It will just monitor your logfile for changes and on any updates, sends the new logs to logstash. Setting up Lumberjack I found compiling lumberjack to be a bit of pain on SUSE(Yes, unfortunately I have to use SUSE). You can have a go yourself by grabbing it from source. You will need to install Go Language to compile it. Luckily I found rpm and deb packages here. Install the deb by dpkg -i debfilename or rpm by rpm -i rpmfilename Set it up as a service - init.d script Set up the config file - sample config. Create the config file at /etc/default/lumberjack It is a good idea to restart lumberjack when you deploy or rotate logs. There is a new protocol for lumberjack v2. I will play with it and update the documentation on that. Key and SSL Certificate for lumberjack You'll notice that key and certificate files are required by logstash and lumberjack. These are required just to encrypt the log shipping. All the commands I used to generate them are consolidated here Kibana This is the easiest part. Just see here Once you have everything setup, you should be able to see logs in the Kibana UI. Play around to see the powerful search feature. You can also write your custom queries for .e.g. to get all requests from an IP which raised an exception just put this query in the search field - @fields.ip:"157.191.96.84" AND @tags:"exception" Inspirations https://github.com/roidrage/lograge http://www.vmdoh.com/blog/centralizing-logs-lumberjack-logstash-and-elasticsearch Loggly and Logentries

Skyline Algorithm - A Binary Tree Approach

over 4 years ago | Shadab Ahmed: Shadab's Blog

If you're into algorithms, you must have heard of this puzzle: Drawing the Skyline A number of buildings are visible form a point. Each building is a rectangle, and the bottom of each buliding lies on a fixed line. A building is specified using a triple of (Left, Height, Right). One building may partly obstruct another, as shown below: Skyline The skyline is the list of coordinates and corresponding heights of what is visible. For example, the skyline to the buildings on the left in figure above is given in the figure on the right. Example input: (1,11,5), (2,6,7), (3,13,9), (12,7,16), (14,3,25), (19,18,22), (23,13,29), (24,4,28) Example output: 1, 11, 3, 13, 9, 0, 12, 7, 16, 3, 19, 18, 22, 3, 23, 13, 29, 0 This puzzle is particularly popular in academia and also as an interview question. If you google it, you would find many research papers as well. Skyline Tree Now I am presenting a binary tree solution - Skyline Tree. This tree is very much similar to a tree I created earlier - Range Count Tree. Here is goes: For the input we are given a set of tuplets containing - where a building starts, where it ends and the height. We use this input to build a binary search tree in such a way, that the nodes in the tree represent the maximum height for a given range of start and end coordinates. Each node in the tree has attributes range_start, range_end and value(height). We just follow these rules when creating the tree: Each parent has a greater value(height) than any of its children When a child is being inserted at a node, it is added in left subtree if the range of the child completely lies to the left of the node's range and vice versa for the right If the incoming range intersects with the current node, we split the incoming range and then add the split parts as children of the node. For e.g. if we already have 4,5 as an existing range and incoming is 2,8 it becomes: 4,5 / \ 2,4 5,8 This is done because the parent 4,5 has greater value(height) than range 2,8 so only the tallest portions are added to the tree, for the ranges of these nodes. Now, how do we ensure that a parent is taller than its children ? We simply insert nodes in the Skyline tree sorted by their height. Time for pretty pictures: Input: (1,11,5), (2,6,7), (3,13,9), (12,7,16) Skyline Tree Partial Starting from the left, you can see, range 1 to 3 has maximum height 11, 3 to 9 has 13 and 12 to 16, 7. Now let's see the tree for the full input: Input: (1,11,5), (2,6,7), (3,13,9), (12,7,16), (14,3,25), (19,18,22), (23,13,29), (24,4,28) Skyline Tree Full The code for SkylineTree is here(gist) and the code to solve the puzzle using the Skyline tree: require './skyline_tree' input = [[1,11,5], [2,6,7], [3,13,9], [12,7,16], [14,3,25], [19,18,22], [23,13,29], [24,4,28]] input.sort!{|x,y| y[1] <=> x[1]} stree = SkylineTree.new input.each do |start_range, value, end_range| stree.add [start_range, end_range], value end stree.print_skyline # output # 1, 11, 3, 13, 9, 0, 12, 7, 16, 3, 19, 18, 22, 3, 23, 13, 29, 0 You can also run the code on CodeBunk The tree we built can be used for similar problems like area under the skyline etc.

Parenthesis Permuation

over 4 years ago | Shadab Ahmed: Shadab's Blog

Another interesting puzzle: Parenthesis Permutation Given N pair of parenthesis. Write an algorithm which would print out all permutations, possible with those parenthesis given that parenthesis are in correct order (i.e. every open parenthesis is matched with closed parenthesis) For .e.g. .. N =3 should give: ()()() (()()) ()(()) (())() ((())) There are recursive solutions for this you can find just by googling. I thought of a non-recursive solution Click to view my solution The basic idea is to construct a Binary Tree, where left node is '(' extra and right node is ')' extra from the current node.  Each node has a weight = number of '(' - number of ')'. We start with a root node '(' at level 0 and create the tree such that the 2N-1 level will contain permutations for N pairs.Now when constructing the tree, we create a child only if:Weight of incoming child is not less than 0When weight >= 0, it should be less than or equal to the numbers of levels to create, so that we have enough parenthesis left to balance the stringLet's take a look at the tree created (string -> weight) for N = 3:An optimization - the last level is not required since all that is added is a ')'Now the code. Infact not using a tree at all, just a linked list and traversing in preorder style, using the concepts above of adding a child. I could have used an array instead of linked list, but it suffers from list expansion frequently hence slowing everything down.The code to generate the tree picture is here Infact the number of permutations for a given N pair form a series called Catalan Numbers . It goes like this: N=1 P=1 N=2 P=2 N=3 P=5 N=4 P=14 N=5 P=42 N=6 P=132 N=7 P=429 N=8 P=1430 N=9 P=4862 N=10 P=16796 N=11 P=58786 N=12 P=208012 N=13 P=742900 N=14 P=2674440 N=15 P=9694845

Transform One String to Another

over 4 years ago | Shadab Ahmed: Shadab's Blog

Another interesting puzzle :- Transform One String to Another Let S and T be strings and D a set of strings. You can say that S produces T if there exists a sequence of strings, SEQ=[s1, s2, ..., sn-1] which meets these criteria: s0 = S sn-1 = T All members of SEQ belong to set D Adjacent strings have the same length and differ in exactly one character. For example, given the set {"cat", "bar", "bat"}, you can say that "cat" produces "bar" by ["cat", "bat", "bar"] Or, given the set {"cat", "red", "bat"}, you can say that "cat" does not produce "red". Given a set D and two strings S and T, write a function to determine if S produces T. Assume that all > characters are lowercase letters. If S does produce T, output the length of a shortest production sequence; otherwise, output -1. Click to view my solution Solving this using a graph. Not the most efficient graph, but it does have pretty pictures :)First let's take this sample data:words = ['simple', 'dimple' , 'pimple','fickle', 'sickle', 'simkle', 'kettle', 'settle'] start_word = 'simple' end_word = 'fickle'Now let's generate a graph, where each node is the word and an edge represents, the index where the words differ and the character that is different.Next, using Djiktra's algorithm, just find the shortest path between the start_word and end_word::Code here(also generates images using graphviz):

Ditch jsFiddle - Codepen is awesome

over 4 years ago | Shadab Ahmed: Shadab's Blog

If you like jsFiddle then you should definitely try Codepen. I am now a Codepen convert for the reasons listed. Reasons why Codepen is better than jsFiddle The most awesome feature on Codepen is live update . No need to save and reload unlike jsFiddle Another awesome feature on Codepen - Compiled preview for both CoffeScript and SCSS. See below: Codepen Preview More options for markup, apart from HTML HTML Options Codepen Fully SCSS compliant. jsFiddle fails to compile SCSS when I added this: .arrow::after{ content: ">"; } Complete error description for SCSS unlike jsFiddle, which silently fails and you have to dig with your browser inspector tool. jsFiddle SCSS Error jsFiddle Error - Silently renders scss as css and just adds an error message in the beginning of the file Codepen SCSS Error Codepen Error - Matching line number and error description For CoffeeScript, Codepen shows you a similar error banner like the scss error but does not show any details. This is atleast better than jsFiddle which fails silently The UI is your preference, but I had no problems switching from jsFiddle to Codepen. Cool Codepens This is a simulator I created for the problem: N Rumors There are n people, each in possession of a different rumor. They want to share the news with each other by sending electronic messages. What is the minimum number of messages they need to send to guarantee that everyone of them gets all the rumors? Assume that a sender includes all the rumors he or she knows at the time the message is sent and that a message may only have one addressee. Each cell in the simulator has the person number in blue and number of rumors with the person in green. If a person is sending out a message his cell becomes pink and the cells for people receiving the message are green. Full code here. Works in Chrome and Firefox. Check out this Pen! Much more impressive codepens: Falling Leaves Tearable Fabric Wall Clock You will find many more cool codepens on their homepage. Happy Codepenning :)

Puzzle - A surpassing problem

over 4 years ago | Shadab Ahmed: Shadab's Blog

Another week, another interesting puzzle. This puzzle is taken from a programming exercise posed by Martin Rem in 1988 A Surpassing Problem By definition, a surpasser of an element of an array is a greater element to the right, so x[j] is a surpasser of x[i] if i < j and x[i] < x[j]. The surpasser count of an element is the number of surpassers. For example, the surpasser count of the string GENERATING are given by G E N E R A T I N G 5 6 2 5 1 4 0 I 0 0 The maximum surpasser count is 6. The first occurrence of the letter E has six surpassers, namely N, R, T, I, N, and G. Rem's problem is to compute the maximum surpasser count of an array of length n > 1 and to do so with an O(n log n) algorithm. Click to view my solution Not the most succint solution, but it does match the O(n logn) criteria. Infact I am using the same RangeCountTree used in the last puzzle.Here's what I am doing (assuming word only contains capital letters):1. Start iterating from the reverse of the string. For the current example, you first see a G so insert a range A,G in the tree. The range A,G in the tree would now have count 1. This just means that surpassers count for letters A to F is 1.  Please note that A,G means letters A to F(G is excluded). 2. Now let's build this tree for NERATING. See it below:The representation is such, that for any new character, the corresponding range in the tree would give the count of surpassers.3. Calculate the max_surpassers while building the tree itself. So now when you see E next after NERATING, the matching range for E in the above tree is E,G -> 6. Therefore the current max_surpassers is now 6Let's see how the tree looks for the whole string GENERATING:Now if the string was changed to REGENERATING, you would see that the current tree has E,G -> 7 , so the max surpassers calculated would be 7We could do the same thing for finding max surpassers in a list of numbers, only difference being that we would insert each range as (-Infinity, current_number)Code here:require './range_count_tree' data = "REGENERATING" rtree = RangeCountTree.new max_surpassers = 0 data.reverse.each_char do |char| # Set range_start to A for the min possible range range_start = ?A.ord range_end = char.ord # Find the surpassers for the current character in the current tree current_max_node = rtree.find(range_end) # set max surpassers if current surpassers is larger max_surpassers = current_max_node.count if current_max_node && current_max_node.count > max_surpassers # Do not add this range to the tree, since we will not encounter any characters in this range # This is just an optimization and not necessary next if range_start == range_end # Update the current tree for the next stream of characters rtree.add([range_start, range_end]) end puts "Max surpassers is #{max_surpassers}" I think there is a much better solution which would put my verbose approach to shame with its simplicity

RangeCountTree - A BST for range intersections

over 4 years ago | Shadab Ahmed: Shadab's Blog

Recently I came across an interesting puzzle: An editor of The History of the World of Science wants to find out the time when the largest number of prominent scientists were alive. The prominent scientists are, by definition, the people mentioned > in the book with the dates of their birth and death. (No living scientists are included in the book.) ? Devise an algorithm for this task if it has the book's index as its input. The entries in the index are sorted alphabetically and give the persons' birth and death years. If person A died the same year >person B was born, assume that the former event happened before the latter one. After giving it a bit of thought, I implemented an interesting data structure to solve it like a basic binary search tree. Here it goes: Here's the tree generated for the data in scientists.rb(using graphviz): tree

MAC OS X 10.8 VPN Connectivity Fix

almost 5 years ago | Shadab Ahmed: Shadab's Blog

I recently updated my MAC OS X version to 10.8.2. After this update, my VPN started having random issues. Sometimes while connecting it would show me authentication failure or no server response while at other times it would connect perfectly. On some networks it refused to let me connect at all. I have found a solution to this and now my VPN connects fine every time. Here's how: Step 1 Open terminal and type: # Create backup cp -r /Library/Preferences/SystemConfiguration ~ # Delete the SystemConfiguration sudo rm -rf /Library/Preferences/SystemConfiguration This will delete all your custom networks. Make sure you have the settings written down somewhere. Step 2 Reboot and recreate the VPN network connection Test your VPN now. Voila! It works !

Introducing MultiConfig gem for ActiveRecord

about 5 years ago | Shadab Ahmed: Shadab's Blog

If you need your model to use a different database from the default databases(production, development etc.) then you have to call method establish_connection in the model. The parameters are either complete database config or a key name as specified in database.yml like other_test or other_production etc. If u need it to dynamically change according to the environment then you write: MyModel < ActiveRecord::Base establish_connection "other_#{Rails.env}" end This messes the database.yml a bit as you end up with a database.yml with configs it should not have, since the other configs are not really environments. This gave me an idea to create a gem which allows you to specify separate config files for models which access other dbs. It's this simple: # In your Gemfile require 'multi_config' # In your model MyModel > ActiveRecord::Base self.config_file = 'other' end Your config file config/other.yml just needs to have similar environments to database.yml. You can see it is much simpler to maintain and cleaner. This is how it works: ActiveRecord stores the configurations read from database.yml in ActiveRecord::Base.configurations hash (copied from Rails.application.config.database_configuration). When establish_connection connection is called with a string argument, it loads the config from the configurations hash having key same as the argument. Multi config extends this hash. It prefixes filename_ as a namespace in the keys to avoid collisions. Then it calls establish_connection with the namespaced key for the current environment. That's the core really; rest of the code deals with railties initialization, error conditions and the majority is tests. I have been absolutely obsessive about tests and documentation. I have added comments to every detail like Bundler.require etc so that it can serve as goto guide for writing gems for myself and maybe for you too. Similarly CHANGELOG and README are created as per best practices. For continous integration I am using travis which is really easy to integrate with github. One interesting thing I did: # In your environments config file (e.g. development.rb) ActiveRecord::Base.config_file = 'other' Now, all your models will use the new config file. Though this can also be done by setting DATABASE_URL environment variable There are some cavieats of course, like migrations. I may add support for those as well in future. Meanwhile you can do this : Migration om different DB I will be glad if you download/install the gem, fork the repo or share this article. Thanks for reading. view source here

Fix for Passenger “unexpected end of file detected”

about 5 years ago | Shadab Ahmed: Shadab's Blog

I recently saw this error on one of our servers – Passenger/Nginx error “unexpected end of file detected.”. This was giving no clue at all in the stack trace on what was the exact problem. Everyone has wracked their brains over it with no solutions. We have an apache/passenger stack with REE 1.8.7. No matter what version of passenger we installed, it was giving the same error. On further digging in the apache logs I saw: *** glibc detected *** Passenger ApplicationSpawner: /opt/apps/current: free(): invalid pointer: 0x0000000002eb1780 *** and somewhere down in stack trace 7f66ac1f8000-7f66ac215000 r-xp 00000000 fd:09 5360931 /opt/ruby-enterprise-1.8.7-2010.02/lib/libtcmalloc_minimal.so.0.0.0 7f66ac215000-7f66ac415000 ---p 0001d000 fd:09 5360931 /opt/ruby-enterprise-1.8.7-2010.02/lib/libtcmalloc_minimal.so.0.0.0 7f66ac415000-7f66ac416000 r--p 0001d000 fd:09 5360931 /opt/ruby-enterprise-1.8.7-2010.02/lib/libtcmalloc_minimal.so.0.0.0 7f66ac416000-7f66ac417000 rw-p 0001e000 fd:09 5360931 /opt/ruby-enterprise-1.8.7-2010.02/lib/libtcmalloc_minimal.so.0.0.0 Reinstall REE I did a quick google on tcmalloc and found out other people also having the same issues. So I installed ree without tcmalloc. REE doesn’t require it but generally performs better. TCmalloc is part of google-perftools which REE includes. It is an improved memory allocator. You may still want try to look into why it’s not working with tcmalloc. To install without tcmalloc just extract the REE archive and run this: ./install --no-tcmalloc Voila! this solved all the issues. Of course I had to whip up a quick bash script to reinstall bundle for all the apps on the server. I will update this post if I make tcmalloc working :) Update: Seems like a version issue. I was using ruby-enterprise-1.8.7-2010.02 earlier. I compiled ruby-enterprise-1.8.7-2012.02 with tcmalloc and i did not face the tcmalloc issue.

Cool JavaScript Tools

over 5 years ago | Shadab Ahmed: Shadab's Blog

I just wanted to mention some of the cool JS tools I came across: Google Closure Compiler – Use this to minify or prettify(unminify) any javascript. It can also reoptimize your javascript with variable renaming etc. Here’s an example of code I unminified(js from facebook): Before function ChatOptions(a,b){this._jslog=JSLogger.create('chat_options');this._jslog.log('server',{vis:a});this.visibility=!!a;this._lastVisibilityChangeTime=0;this.settings=b;} After function ChatOptions(b, a) { this._jslog = JSLogger.create("chat_options"); this._jslog.log("server", {vis:b}); this.visibility = !!b; this._lastVisibilityChangeTime = 0; this.settings = a } Note that the compiler even rearranged the arguments list. Similarly you can compress javascript code using Clojure, which generates logically same code, however it will be more optimized for saving space. Google has provided full api for clojure compiler which you can see here JSFiddle : This is an online environment where you cam create HTML, JS and play around with it. It even supports adding jQuery, prototype and other popular js libraries. What’s even more cool ? You can even save the snippets and share with other people, just like Pastebin or Github Gist. They also have great examples you can see to get started. So have fun and happy coding :)

Tuning up Heroku

over 5 years ago | Shadab Ahmed: Shadab's Blog

So now you have deployed your site on Heroku, it’s time to make it fast, responsive, backed-up and analytics enabled. With Heroku you can do all these thing for free .. yayyy (atleast at the time of writing this blog post) Solve Heroku Startup Delay If you are using Heroku free for your RoR(Rails 3.0) app you might be noticing long startup delays. This happens because if your site has fairly low footfall, which it would obviously have when you have just launched your app. What happens is that heroku unloads your app if there is no activity for some time. You can very easily simulate traffic to your site and keep the heroku dyno(app engine) active by adding a cron on any machine which runs 24/7: #cron entry for wget. Every 15 minutes */15 * * * * wget http://www.yoursite.com/ -o /dev/null I have noticed site load time fall down to 1.5s from 10-20s when accessing the site after a while. Add Caching First you have to enable caching for your app. See here to add caching to you app. Now, go to your app folder and the run command below to install the free memcache addon to your app: heroku addons:add memcache Now, in your Gemfile, add – gem 'dalli' Add this line in your config/environments/production.rb – config.cache_store = :dalli_store Now run: bundle update git commit -a -m 'Added memcache' git push heroku master Thats it. Your site is now cache enabled. No other configuration required. To test the cache status just run these commands after a few page hits: heroku run console > Rails.cache.status For more information see – Heroku Memcache Enable Backups Creating backups on heroku is also very easy. You can take manual backups and also enable automated backups, all for free :). Add backup addon by – heroku addons:add pgbackups:auto-month. The auto-month addon gives you maximum auto and manual addons for free. Heroku states for this addon – Daily automatic backups. Retains 7 daily backups, 5 weekly backups, and 10 manual backups . Your automatic backups will be done by Heroku. If you do need to create a manual backup, you can do by - heroku pgbackups:capture SHARED_DATABASE (DATABASE_URL) ----backup---> b252 Capturing... done Storing... done The value b252 is the backup id. To list all backups: heroku pgbackups ID | Backup Time | Size | Database -----+---------------------+---------+---------------- b001 | 2012/02/19 06:34.29 | 24.5KB | SHARED_DATABASE a001 | 2012/02/20 20:00.55 | 24.5KB | DATABASE_URL b004 | 2012/02/21 22:36.57 | 24.5KB | SHARED_DATABASE a004 | 2012/02/23 20:00.55 | 24.5KB | DATABASE_URL Get download link for any backup: heroku pgbackups:url b001 #use backup id from backup list . Download the dump by curl backup_url Restore the dump to you local database using pg_restore --verbose --clean --no-acl --no-owner -h myhost -U myuser -d mydb latest.dump. Make sure that the user specified is the owner of both db and the public schema inside it. Single line backup/restore: heroku pgbackups:capture && curl `heroku pgbackups:url` | pg_restore --verbose --clean --no-acl --no-owner -h myhost -U myuser -d mydb To get more information see – Heroku pgbackups New Relic – Site Analytics Addon To analyze your site performance, heroku supports New Relic. This is also very simple to add to your app. First you just need to enable New Relic for your app – heroku addons:add newrelic:standard. This is again free :) Now in your app, add this like to your Gemfile – gem 'newrelic_rpm'. Now copy this file newrelic.yml, and save it in your config directory. Then do the usual steps of committing and pushing to Heroku and you’ll be done. To view New Relic stats, just goto your app page on heroku, click on Addons, you will see New Relic in the list. Click on that link to open the awesome dashboard like shown below:

Setting up your RoR app on Heroku

over 5 years ago | Shadab Ahmed: Shadab's Blog

If you are looking to host your rails site for free(or even paid) without worrying about server configuration etc , think no further than Heroku. Heroku is one of the most awesome cloud platforms to host your app with minimal fuss. You can find many guides and quick starts for heroku but you may still spend hours looking at multiple guides. So if you are set out to do exactly what I wanted to do i.e. Deploy my app on heroku at my domain, this blog will be helpful. Tested on Ubuntu 11.10 Setting up git First we need to setup your application’s git repository. Ignore if you have already set this up. git init touch .gitignore git add . git commit # This will create a local repository Install and Setup Heroku sudo apt-get install heroku-toolbelt heroku login # login with your heroku username and password heroku create --stack cedar # create heroku app for the local app. This is for rails 3.1+. Do this in your app directory Verifying Heroku Setup Confirm your app has been created by – git config -l. Output should be like: user.name=Shadab Ahmed user.email=*******@gmail.com core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true remote.heroku.url=git@heroku.com:empty-samurai-3618.git remote.heroku.fetch=+refs/heads/*:refs/remotes/heroku/* Heroku will become just like another remote repository for you app where you can push commits. Deploying on Heroku Deploying is as simple as a git push. git commit -a git push heroku master If successful you will get your app url you can visit to see the app. You can also rename your app with heroku rename newappname. Now your app will be available on http://newappname.herokuapp.com Setting up your domain for your app If you want to access your app through http://yourdomain.com instead of http://appname.herokuapp.com. Setting up your domain is very simple: heroku addons:add custom_domains heroku domains:add yourdomain.com # Setup your domain for your current heroku app Setup A-Records for your domain for the following IPs: 75.101.163.44 75.101.145.87 174.129.212.2 Also setup C-Name record for your domain www.yourdomain.com appname.herokuapp.com That’s it. You app has been setup and you can access it using http://yourdomain.com . You might want to optimize your site after this. See my post for setting up backups, analytics and performance tuning of your app on Heroku – Tuning Heroku Good Luck! P.S. Heroku free account only supports postgres database as of now. I would recommend using postgres for your test, development and production environments for the sake of consistency

My Enki Fork - An RoR blogging engine

over 5 years ago | Shadab Ahmed: Shadab's Blog

I have been thinking of setting up my blog for a long time. However, I did not want to host a generic wordpress/blogspot blog. Wanted something which I have more control over and I can add features over time. Thats how I ended up with enki . It’s a very simple rails blogging engine which you can customize very easily. Here’s what I changed from the original source: It’s a complete portfolio site and not just a blog anymore. You can add description, cv and projects list. You blog will be under /blog scope. Styles added from Twitter Bootstrap project Performance improvements and analytics to it using Dalli(memcache) and New Relic. Social media bar for each post. Facebook and Google plus right now Disqus comments Changed db to postgres from sqlite for deployment at heroku (Heroku only supports postgres for free accounts). You can see my github fork at : https://github.com/shadabahmed/shadabahmed.com. Follow the steps below to setup your enki clone on Ubuntu 11.10 Install git and checkout code: sudo apt-get install git git clone https://github.com/shadabahmed/shadabahmed.com.git enki Install Postgres database: sudo apt-get install postgresql sudo -u postgres psql postgres Change password to “postgres” – postgres-# \password postgres Exit psql shell – postgres-# \q Create databases: sudo -u postgres createdb enki_test sudo -u postgres createdb enki_dev sudo -u postgres createdb enki_prod Install modified Enki App: Goto app folder and run: cd enki bundle install bundle exec rake db:migrate To customize the blog: Edit config/enki.yml Change title Change the open_id setting under author. Open ID is be an url which you can get from your Open ID provider. You might already be having it. See this Now test the app locally – rails s Open http://localhost:3000/admin and enter your Open ID url. This will ask for authentication and you are through. You can also skip this in development mode. If you wish to host this at heroku, see my post on setting up RoR app at heroku – here