I’m so popular ^_^

It’s been almost a year since I last logged in here, mostly because I have been busy working heads down on some projects. The first thing I noticed is that I have 14,798 pending comments! Most of them are probably spam and it’s funny realizing that this means I received over 40 spam comments per day over the last year. With a zero percent success rate, the spammers are still hopeful?

Access Squid proxy HTTP traffic using the url_rewrite_program option

Squid is one of the most well known caching proxy. It works as a proxy server for protocols such as HTTP and FTP, but utilizes caching when possible to greatly improve content delivery. When I first came across Squid, one of the things I became curious about is how to access and modify HTTP traffic, more specifically just the URLs requested, in real-time. An use case for this is logging to a database for auditing.

Quick research led me to something called the “url_rewrite_program”, which is simply an option available in the Squid configuration file. The path for that is typically something like this:

/etc/squid/squid.conf

The way this option works is that in the config file, you simply add a line like the following where that option name is followed by the path to your rewriter:

url_rewrite_program /home/aktarer/squid/rewriter.php

What Squid does on start is that it spawns several instances of this program (defined by the option “url_rewrite_children”). If you’re using a script file like I am, make sure you set the executable mode. You can do that with a command like the following:

chmod +x /home/aktarer/squid/rewriter.php

As Squid gets requests, it pipes data in the following format to the standard input of your program:

URL <SP> client_ip “/” fqdn <SP> user <SP> method [<SP> kvpairs]<NL>

Your program can simply be a loop waiting on this. Here’s an example of what you might get:

http://www.bing.com 76.249.74.174/- – GET – myip=109.236.81.250 myport=3128

You can parse this like any other string and do as you please like insert to your database. You’ll notice that simply reading input data might cause your proxy to no longer work. Something I didn’t mention earlier is that Squid watches your standard output and requires a response for every input. So when you get the above line as input, you are expected to output something like the following as output, followed by the new line character.

http://www.bing.com

Now let’s say you simply can’t allow people using your proxy to access something like Bing, what can you do? Well instead of outputting the above for such requests, you can output the following:

301:http://www.google.com

This will cause a redirect to Google.

Now you might be wondering, how can you access more information such as the request body? As far as I know, there’s no easy way to do this. However, Squid is open source! This means, you can build your own version to do just that.

Use cases for SSH tunneling (port forwarding)

SHH tunneling, as you might know, allows for transferring data through an encrypted channel. The data being transferred itself may or may not be encrypted. Setting up these tunnels is actually an easy process. But let’s briefly discuss how SSH works first.

SSH stands for “Secure Shell” and is simply a protocol for secure communication. This is commonly used by people to access their servers. On the server side, there is a program like OpenSSH that listens on port 22. On the client side, there is a program such as PuTTY that connects to the server through the given port. The two authenticate through means such as passwords and/or keys. Once this is done, the client is given access to the default shell for the target user and any communication that occurs is encrypted with what’s defined in the SSH protocol.

Back to SSH tunneling. The three types are dynamic, local, and remote. To help you understand the usefulness of each, I will go through three example use cases.

Dynamic

Facebook is blocked at my school. I WANT FACEBOOK!!!

Let’s make this more generic and say you want to access any blocked website on your local network. Being blocked essentially implies that your network activity is being monitored. With the help of dynamic tunneling, you can access any blocked content (provided your server has access to it), and pretty much conceal your network activity to the point where the only thing that can be known by your network administrator or an adversary is that you’re having communication through the SSH protocol.

So how can you can you set up these? In your client program, look for something like “Tunnels” or “SSH port forwarding”. In PuTTY, you’ll find this section in Connection -> SSH -> Tunnels. Under “Add new forwarded port”, select “Dynamic”. For the IP type, leave at “Auto”. For the “Source port”, put down a port that you computer is most likely not utilizing like 32568. The port is limited to 16-bits, meaning the maximum number is 65535. For the destination, leave empty as we want this forwarding to happen through the local interface (i.e. localhost). Click on “Add” and connect normally.

Once you’re done, open up your browser such as Chrome and head over to the settings page. Look for the proxy connection section and set the Socks to be localhost:32568. You can now browse the web securely…well at least like your server and its network.

What if you want to secure the network activity of any application, one that may not support setting the Socks server? Well for that, Google “socksifier” and pick one.

Local

I want to access VNC running on my server securely!

VNC (Virtual Network Computing) is a popular way of accessing your computer desktop remotely. It works through the RFB (Remote Framebuffer) protocol, which is NOT a secure protocol. Let’s say that you have a window system such as X running on your Linux server. If you want to access it through the network on another computer, what you might do is run a program on your server such as TightVNC. TightVNC server will listen on a port like 5901, on maybe the LAN interface like eth0. On your main computer, you’ll run TightVNC client to connect to the server given the network address and port. After maybe authentication, you’ll essentially have access to your server’s desktop! The problem is, what if you’re main computer is on a network that can easily be monitored?

An adversary can easily capture and possibly decrypt your VNC password. However, that’s not all. The data being sent for you to be able to control the server’s desktop is not encrypted. This effectively means the adversary can monitor and maybe even modify what occurs in your VNC session. Not good.

Thankfully, with SSH local port forwarding, you can rest assured of privacy. First what you should do is have the TightVNC server listen on the local interface. A command such as the following should do the trick:

tightvncserver -nolisten tcp -localhost :1

What this does is tells TightVNC to listen on port 5901 for the local interface. This means you will no longer be able to VNC through simply the server’s address and that port from your main computer.

Now that you have this set up, go to your SSH client and find the tunneling section again. Set the type to be “Local”. For the “Destination”, set it to be “localhost:5901″. For the “Source Port”, set it to something like 5910. Add it and connect as usual. Now, to access the server’s desktop, in your VNC client, set the target to simply be “localhost:5910″. You should notice that you are able to connect like before, only now you’re doing it very securely!

Remote

I’m at college. How can I allow my friend from his college network to access an application running and accessible to only my college network?!?

If you consider yourself to know anything about college networks, you’ll know that firewalls play a crucial role. In my college network for example, incoming connections from outside the network are blocked by default for most clients. The only way to go around this is to ask for a firewall exception. However, for the sake of this example, let’s say you don’t have that privilege and neither does your friend. So how can you get this to work?

Well since neither you nor your friend can contact each other directly, one option is to have a third node that’s accessible by both you and your friend. For example, that Linux server you have lying around.

Let’s say the application you want your friend to access is accessible by you through the following address “129.161.72.216:2000″. Go back to your SSH client and find the tunneling section. This time, set the type to be “Remote”. For the “Destination”, set it be “129.161.72.216:2000″. For the “Source Port”, set it to something probably not being used on your server like 5329. Add this and connect as usual.

Your friend should now be able to access the application through YOUR_SERVER_ADDRESS:5329. Awesome! If this does not work, the most likely reason for it is that the OpenSSH server running on your server is listening on port 5329 for only the local interface. To change this, open up your SSH daemon config file, which is probably the following:

/etc/ssh/sshd_config

Look for “GatewayPorts”. If it exists, set it to “yes”. If it doesn’t, simply add it. It should look like the following:

GatewayPorts yes

Save and restart the SSH server. Connect again with your SSH client while making sure the port forwarding is still set. This should now work!

 

Note (s):

Don’t have a Linux server? No problem. Amazon will give you one for FREE! Check out AWS Free Usage Tier.

The best way to audit your Facebook lists (bookmarklet)

One of the most important features of Facebook is ability to have lists. Lists, in case you don’t know, is like Google+’s Circles. They allow you to group your social connections. This means efficient management of who sees what content. I use lists a lot and if it weren’t for them, I probably wouldn’t be using Facebook.

Recently I visited a friend’s profile and noticed that he wasn’t on any of my lists. With no list membership, I had little to no content available for him. If I remember correctly, I did assign lists to him. Perhaps the network requests didn’t go though? Perhaps Facebook’s database write failed? Who knows…

One thing’s for sure, it was luck that led me to discover this issue. I wondered, why isn’t there an easy and quick way to see which lists friends are in without having to check one by one? Facebook does provide a way to see all friends associated with specific lists. However, considering people have hundreds to thousands of friends, it’s not practical to find people that aren’t correctly associated.

I wanted to solve this problem…so I quickly put together a bookmarklet. A bookmarklet acts like a bookmark, which just runs some JavaScript code. My code works only with your browser and Facebook’s servers, so you can rest assured of your privacy. Just create a bookmark and set the url to be this: facebook lists audit. You might be able to just drag and drop this link to your bookmark bar. Then visit this url and click on the bookmarklet.

If you’re interested in seeing how it works, here’s the uncompressed version.

How to get almost every visitor to like your website (Facebook)

Do you like cats? Do you hate cats? Whatever the case, I want you to click on the above picture. Once you do, I will explain why I am asking you to do this. (If my explanation doesn’t appear below on the first click, wait a bit and click again.)

* trick no longer works so content below is automatically revealed

Since you are seeing this, most likely it means that you followed my directions. What do you think is the percentage of people that are seeing this who didn’t click on the cat? Probably low since it is easier to click than to edit a style property. Now what do you think is the probability that any visitor who came to my post, clicked somewhere? It’s probably very high. You might be wondering, why am I mentioning clicking so much? That’s because I am going to show you that all it takes for a visitor to like your page is a click…any click…anywhere on the page.

Facebook offers a social plug-in for websites called “Like Button”. What this does, if you don’t already know, is that it creates a button on a website and that allows facebook users to “like” with just a click. This is great because the “like” registers as a post on their facebook profile, opening the opportunity for lots of referral traffic.

A website owner can add the like button to their website in three ways. They are HTML5, XFBML, and IFRAME. These are different in what they can offer, but similar in what they do for the like button. To use the IFRAME method, the website owner simply has to copy a static iframe tag to his website. The other two methods are extensions to this idea. They use JavaScript to dynamically create the iframe.

Using iframes for such applications is a good choice because in many modern browsers, security protocols are in place that prevent Cross-site Scripting (XSS), for example, without getting in the way of features pertaining to the iframed webpage (i.e. scripts from a webpage won’t know much about an iframe’s content if it comes from another website). In the case of the like button, this makes it much harder for websites to get users to like something without their consent.

An obvious solution is to have the like button iframe covertly follow the mouse. By doing this, when the users click, they click on the like button. The problem is that the site owner doesn’t get feedback on happens inside the iframe. This means he has to blindly remove the iframe or his users’ ability to interact with his site becomes limited. This exemplifies why using iframe is a good choice. What do you think is a better: having lots of likes but frustrated users or decent number of likes and satisfied users? The latter is probably better in the long run.

However, since the HTML5 and XFBML do not rely on just an iframe, they are more susceptible to malicious intents. Today I am going to present a way for someone to achieve lots of likes and still maintain satisfied users when using the HTML5 or XFBML method. I can get almost any Facebook user/visitor to like something with just a click anywhere on a page without the user being suspicious.

If you go to this page, you will see a like button. That like button is created using the HTML5 method and the target is the URL to this post. For the page to load the way it does, the generated source has to change. The Facebook code causes your browser to load JavaScript directly from their servers, which ends up adding styles and more. It also creates an iframe tag to the like button and its other features. Wait, so where’s the hack if it creates an iframe?

If you recall, I said that scripts from one website can know much about an iframe’s content if it comes from another website. However, if it comes from the same website, the scripts have much more power. I also said that the HTML5 and XFBML methods change the generated source for a webpage. This all means that you can detect changes! By detecting changes, you can dramatically improve your odds of removing a trailing iframe to still achieve your desired goals.

A working clean demo is available here. The way it is set up is that it first detects if the visitor is logged into facebook. It does this through a hack revealed here. If you are not logged in, it removes the trailing iframe. If it doesn’t do this, a Facebook sign in popup will show up upon click, which will undoubtedly make users suspicious. If you are logged in, the trailing iframe is left enabled and it points to the html page with the HTML5 like button. That iframe is set as 2px by 2px, with 0.01 opacity, so it’s definitely hard for a user to notice. At the same time, the script monitors for changes in iframed source every tenth of a second. I have found that the source changes more than two times when a user hits like.

When it arrives at that state, we can make the assumption that the user clicked like. The script then creates a cookie and removes the iframe. When the user refreshes with the cookie still present, the iframe is not shown. The removal allows the user to resume normal activities.

A flaw in this design is that we don’t know if a user removes the like directly from Facebook. If the user does that and the cookie is present, he will not be presented with the iframe again. Another flaw is that if the user removes the cookie while still having the liked state, he can no longer do click events on the webpage. Obviously the latter is what we mostly want to avoid. As far as I know, there is no way to figure out the true like status of a user. Maybe instead of creating just a standard browser cookie, it should create an evercookie?

Now imagine the potential. Let’s say that every user that “likes” a page brings in two referrals new to the page. That means by the nth level, where n starts with 1, there would have been 2^n – 1 new visitors. When n = 30, the number of visitors will be 1,073,741,823!

This trick works very well on Google Chrome. It sometimes works on Firefox and Internet Explorer.

The solution to this problem is to prevent the source generated by the HTML5 and XFBML methods outside the iframe from changing so much. This will render the trick futile. Site owners would then again find the trailing iframe not appealing given the drawback of no feedback. The Twitter follow button is an example of what I am talking about.

RPI Course Alerter – Get Notified Via SMS!

At RPI (Rensselaer Polytechnic Institute), it seems inevitable that you will come across a course you want to take and then be left in dismay when you notice that all the slots are taken. The good thing is, students do drop courses with time. However, you’re then competing with everyone else that wants the slot in question. To give you the edge, I developed a tool to send you a text message within a few minutes of someone dropping a course. So let’s walk through how it works.

Continue reading RPI Course Alerter – Get Notified Via SMS!

My experience as the intern at AppNexus

What’s AppNexus? What’s Real-Time Bidding (RTB)? What’s up with all the orange and black? These were among the many questions I had when I first came across AppNexus.

This past summer, I was the software engineering intern at AppNexus. I landed the role thanks to a program called the NYC Turing Fellowship Program, which aims at matching students in North America with leading startups in NYC. Before this program, I never even heard of AppNexus or ever thought I would be working for such an advertising company. Looking back, I realize I have no regrets in choosing this path. AppNexus never failed to amaze me, so allow me to tell you why.

http://techblog.appnexus.com/2011/my-experience-as-the-intern-at-appnexus/

musings of a techie