{"id":157,"date":"2011-08-13T19:15:42","date_gmt":"2011-08-14T00:15:42","guid":{"rendered":"http:\/\/trillworks.com\/nick\/?p=157"},"modified":"2012-04-10T23:46:30","modified_gmt":"2012-04-11T04:46:30","slug":"pillowchat-how-not-to-build-a-chat-room-with-jquery-phpillow-and-couchdb","status":"publish","type":"post","link":"https:\/\/trillworks.com\/nick\/2011\/08\/13\/pillowchat-how-not-to-build-a-chat-room-with-jquery-phpillow-and-couchdb\/","title":{"rendered":"PillowChat: How Not to Build a Chat Room with jQuery, PHPillow, and CouchDB"},"content":{"rendered":"<p>After watching J. Chris Anderson show off a CouchDB chat app at an Austin Javascript meeting, I figured Couch might be a good fit for my next large project. Building a clone of my own would be a good way to get familiar with the tech. Since I knew my back end would be PHP, I opted for\u00a0Kore Nordmann&#8217;s <a href=\"http:\/\/arbitracker.org\/phpillow.html\">PHPillow<\/a> wrapper after reading some good things on StackOverflow.<\/p>\n<p>Here&#8217;s the result:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-181 alignnone\" title=\"PillowChat Screenshot\" src=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/pillowchat-300x295.png\" alt=\"\" width=\"300\" height=\"295\" srcset=\"https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/pillowchat-300x295.png 300w, https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/pillowchat.png 618w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/p>\n<h2>Structural Overview<\/h2>\n<p><strong>Client: <\/strong>jQuery runs in the browser, sending messages and polling the server for new ones. Application settings and state are maintained in the global pillowchat object. Functions beginning with &#8220;render&#8221; read state information from the global and push it into the DOM.<\/p>\n<p><strong>Server: <\/strong>PHP receives POST requests from the client and handles them in chat.php, sending back JSON messages to the client. There are three CouchDB views defined in views.php for performing the following actions:<\/p>\n<ul>\n<li>Getting a list of users<\/li>\n<li>Getting recent messages<\/li>\n<li>Getting the timestamp on a user&#8217;s empty message<\/li>\n<\/ul>\n<h2>Implementation Specifics<\/h2>\n<p>CouchDB only stores &#8220;message&#8221; documents. They contain three properties: Username, tripcode, message, and timestamp. Each message includes a password. The server hashes this into a tripcode that allows users to identify each other without registration. Only the hashes are saved.\u00a0Mouse over usernames in the chat room to see the tripcodes.<\/p>\n<p>Rather than create a separate document to keep track of users&#8217; most recent activity, I just record timestamps in an empty message document. Every time the client polls for new messages, the empty message document for that client is updated with a timestamp. The getUsers function grabs a list of active users by selecting all the empty message documents with timestamps in the last 5 seconds. Since the views are predefined, you can&#8217;t change the timestamp in the request sent to CouchDB. Instead, I return all empty messages and use PHP to loop through the response, looking for the most recent. It seems like there should be a better way to do this.<\/p>\n<p><span class=\"Apple-style-span\" style=\"color: #000000; font-size: 22px; line-height: 32px;\">The Stress Test from Hell<\/span><\/p>\n<p>The app looked nice and appeared to work pretty well when I roped in some late night facebook lurkers to test it. Feeling confident in my creation, I showed it off to my neighbor <a href=\"https:\/\/twitter.com\/#!\/kmobs\">@Kmobs<\/a> while picking up some cookies from his apartment. The app was in no way prepared for his hordes of CyanogenMod followers. The chat appeared to work okay with about 25 people in the room, but became inaccessible almost immediately after.<\/p>\n<p>All the polling clients hammered the server, maxing out the Linode instance&#8217;s CPU and memory.<\/p>\n<p><a href=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/apache.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-171\" title=\"Apache Overload\" src=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/apache-227x300.png\" alt=\"\" width=\"227\" height=\"300\" srcset=\"https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/apache-227x300.png 227w, https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/apache.png 586w\" sizes=\"auto, (max-width: 227px) 100vw, 227px\" \/><\/a><\/p>\n<p>It&#8217;s obvious that something with an event loop like Node.js would have been far more appropriate here. Apache&#8217;s processes quickly wiped out the RAM.<\/p>\n<figure id=\"attachment_173\" aria-describedby=\"caption-attachment-173\" style=\"width: 300px\" class=\"wp-caption alignnone\"><a href=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/bandwidth.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-173\" title=\"bandwidth\" src=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/bandwidth-300x138.png\" alt=\"\" width=\"300\" height=\"138\" srcset=\"https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/bandwidth-300x138.png 300w, https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/bandwidth.png 829w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><figcaption id=\"caption-attachment-173\" class=\"wp-caption-text\">There goes 1\/3 of my monthly bandwidth.<\/figcaption><\/figure>\n<p>Linode provides some nice graphs of <a href=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/cpu.png\">CPU<\/a>, <a href=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/bandiwdth.png\">bandwidth<\/a>, and <a href=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/io.png\">disk IO<\/a>. They all confirm that the server was swamped. I&#8217;m curious to see how a static web page or WordPress install would stand up to this kind of traffic.\u00a0The trouble didn&#8217;t stop there. During the post mortem I saw that the database size had ballooned to a whopping 5.8 GB.<\/p>\n<p>What had I done wrong? Had I stored the message documents in an inefficient way? Was there some kind of bug causing duplicate documents? Probably not. Here&#8217;s what I saw when I wrote a view to dump out all the messages:<\/p>\n<p><a href=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/spam.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-176\" title=\"Chat spam\" src=\"http:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/nyanspam.png\" alt=\"\" width=\"571\" height=\"298\" srcset=\"https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/nyanspam.png 571w, https:\/\/trillworks.com\/nick\/wp-content\/uploads\/2011\/08\/nyanspam-300x156.png 300w\" sizes=\"auto, (max-width: 571px) 100vw, 571px\" \/><\/a><\/p>\n<p>Some crafty hackers correctly assumed there was no rate limiting and flooded the chat with long spammy messages. The easiest way to do this is probably Chrome&#8217;s javascript console. Some sort of shell script would have also worked.<\/p>\n<p>5,939 MB \/ 35,800 Documents = 169.9 KB per document<\/p>\n<h2>The Takeaway<\/h2>\n<p>You already know that you shouldn&#8217;t trust clients. This truism is an understatement. You should write your server side code with the assumption that your client code is being manipulated by devious bastards. In this case, the server failed to verify message length and message frequency.<\/p>\n<p>Several improvements could make this app scale better. \u00a0Websockets are probably better than having clients poll the server every second. When enough people join the chat, the server basically experiences a DDOS attack. Since Websockets don&#8217;t really enjoy wide support, the polling method could still be employed in a more conservative manner. Clients could reduce their poll frequency as an inverse function of active users or perform an exponential backoff when their requests time out.<\/p>\n<p>The PHPillow literature base is pretty small. There is a short tutorial on the official site, but it doesn&#8217;t cover very many use cases. While the API itself is decently documented, but some more examples would go a long way.<\/p>\n<p>When you create a view in PHPillow, it is stored to CouchDB the first time you execute the code. If you want to modify the view, you must <a href=\"http:\/\/tracker.arbitracker.org\/phpillow\/issue_tracker\/issue\/11\">delete the original<\/a> view in Futon before the changes are stored. This is not a big deal, but it&#8217;s frustrating if you don&#8217;t know about it. Additionally, I&#8217;m not thrilled by the prospect of writing a view every time I want to construct a new type of query. CouchDB is good at selecting ranges, but it&#8217;s not immediately apparent how I should locate a document based on 2 string properties, e.g. &#8220;firstname=bob&amp;lastname=loblaw&#8221;.<\/p>\n<p>The PillowChat <a href=\"https:\/\/github.com\/NickCarneiro\/PillowChat\">source<\/a> is available on GitHub. It would be fun to see what kind of volume is possible if the aforementioned improvements are implemented. Big thanks to Keyan and the CyanogenMod crowd for the testing.<\/p>\n<p>Edit: You can build a more efficient chat app with far less code using socket.io and node.js. See <a href=\"https:\/\/github.com\/NickCarneiro\/SocksChat\">SocksChat<\/a>\u00a0for a simple example.<\/p>\n<h2>Resources<\/h2>\n<p><a href=\"http:\/\/wiki.apache.org\/couchdb\/HTTP_view_API\">http:\/\/wiki.apache.org\/couchdb\/HTTP_view_API<br \/>\n<\/a><a href=\"http:\/\/wiki.apache.org\/couchdb\/View_collation#View_Collation\">http:\/\/wiki.apache.org\/couchdb\/View_collation#View_Collation<br \/>\n<\/a><a href=\"http:\/\/arbitracker.org\/phpillow\/tutorial.html\">http:\/\/arbitracker.org\/phpillow\/tutorial.html<br \/>\n<\/a><a href=\"http:\/\/tracker.arbitracker.org\/phpillow\/api\/view\/Core\/phpillowDateValidator.html\">http:\/\/tracker.arbitracker.org\/phpillow\/api\/view\/Core\/phpillowDateValidator.html<\/a><br \/>\n<a href=\"http:\/\/tracker.arbitracker.org\/phpillow\/issue_tracker\/issue\/11\">http:\/\/tracker.arbitracker.org\/phpillow\/issue_tracker\/issue\/11<\/a><\/p>\n<h2>Miscellany<\/h2>\n<ul>\n<li>Styling is borrowed from <a href=\"http:\/\/tracker.arbitracker.org\/phpillow\/issue_tracker\/issue\/11\">J\u00e9r\u00f4me Gravel-Niquet<\/a>&#8216;s backbone.js demo.<\/li>\n<li>I have no idea what will happen if you open the demo in Internet Explorer.<\/li>\n<li>I changed the name from PillowTalk to PillowChat after I realized someone else on github beat me to the pun.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>After watching J. Chris Anderson show off a CouchDB chat app at an Austin Javascript meeting, I figured Couch might be a good fit for my next large project. Building a clone of my own would be a good way to get familiar with the tech. Since I knew my back end would be PHP, &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/trillworks.com\/nick\/2011\/08\/13\/pillowchat-how-not-to-build-a-chat-room-with-jquery-phpillow-and-couchdb\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;PillowChat: How Not to Build a Chat Room with jQuery, PHPillow, and CouchDB&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[77],"tags":[32,31,35,34,14,33,36],"class_list":["post-157","post","type-post","status-publish","format-standard","hentry","category-technical","tag-chat-room","tag-couchdb","tag-javascript","tag-jquery","tag-php","tag-phpillow","tag-stress-test"],"_links":{"self":[{"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/posts\/157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/comments?post=157"}],"version-history":[{"count":25,"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/posts\/157\/revisions"}],"predecessor-version":[{"id":503,"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/posts\/157\/revisions\/503"}],"wp:attachment":[{"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/media?parent=157"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/categories?post=157"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/trillworks.com\/nick\/wp-json\/wp\/v2\/tags?post=157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}