tag:blogger.com,1999:blog-3715000527290050953.post4858505216762745994..comments2023-04-03T23:50:30.746-07:00Comments on Fun Anymore [Retired]: Help needed on TwitterEvo Terrahttp://www.blogger.com/profile/12465633247738209287noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-3715000527290050953.post-92113422273957066392007-04-06T04:19:18.000-07:002007-04-06T04:19:18.000-07:00Heya - you don't know me from Adam, but I'...Heya - you don't know me from Adam, but I've been listening to a handful of your podcasts for a year or so now and have dropped comments here or there. Also, I've written a book on Hacking RSS and Atom. (That's the title.)<br><br>Since Twitter doesn't really (yet?) have a way to send out notifications based on arbitrary keywords, what's probably the only solution is to script up a feed fetcher that polls the Twitter feed at a rate of once every few seconds to catch updates before they fall off the end of the feed. The rate is entirely dependent upon just how must usage Twitter gets as a whole and how fast updates travel through that 20 update window.<br><br>The problem with that though, is that Y! Pipes for sure won't do that for you, and the folks at Twitter itself might notice the huge number of hits (as many as 20-80k a day for a hit every few seconds) from a single address and decide to block access for suspicion of launching a denial of service attack.<br><br>That all said, though - once you do have a constant stream of tweets flowing by, it's not so hard to match on keyword or whatnot and then deliver the results in pretty much any form you want.l.m.orchardhttp://decafbad.com/blog/noreply@blogger.comtag:blogger.com,1999:blog-3715000527290050953.post-65514026622682908362007-04-06T04:39:07.000-07:002007-04-06T04:39:07.000-07:00Actually, now that I look at the feed, it's wo...Actually, now that I look at the feed, it's worse than I thought. The Twitter public timeline feed is *not* a live firehose - it seems only to change every 60 seconds or so, from a rough check I just did. So, if you fetch it repeatedly over the course of a minute or so, it never changes.<br><br>So, at worst, that means it's just a snapshot of the last 20 updates from the last 60 seconds - possibly missing a bunch. At best, maybe it means they save up 20 updates before changing the feed, and there just happen to be about 20 updates a minute from the world. (Though, somehow, I doubt that last one.)<br><br>What this means is that, fetching the feed once a minute, you'd have a good chance of catching some tweets with your keywords but would be likely to miss a bunch if the site gets more than 20 updates a minute.l.m.orchardhttp://decafbad.com/blog/noreply@blogger.comtag:blogger.com,1999:blog-3715000527290050953.post-55527031264229285572007-04-06T06:20:35.000-07:002007-04-06T06:20:35.000-07:00I'm not sure you can do exactly what you want,...I'm not sure you can do exactly what you want, but using the <a href="http://groups.google.com/group/twitter-development-talk/web/api-documentation" rel="nofollow">twitter API</a> you might be able to at least pull your follower list, and then pull the feeds of your followers (I'm assuming that people who don't follow you won't address tweets to you).<br><br>You can still only pull 20 hits per person, but it's probably a bit closer to what you want.<br><br>JayJason Penneyhttp://jasonpenney.net/noreply@blogger.comtag:blogger.com,1999:blog-3715000527290050953.post-45190475556037050792007-04-06T08:13:04.000-07:002007-04-06T08:13:04.000-07:00@Jason: Oh, hey, now that's a good idea! Don...@Jason: Oh, hey, now that's a good idea! Don't drink from the firehose, but from the followers' feeds. You won't get the world that way, but you'll be very likely to get most of the part that cares about you.l.m.orchardhttp://decafbad.com/blog/noreply@blogger.comtag:blogger.com,1999:blog-3715000527290050953.post-3600316665053607462007-04-06T10:34:56.000-07:002007-04-06T10:34:56.000-07:00Yes, the twitter API leaves much to be desired. T...Yes, the twitter API leaves much to be desired. The 20 updates per minute is very limiting. For instance, the twittermap misses many updates because there is rarely 20 or fewer updates per minute on the public timeline.<br><br>Seems like the only way to do it within the current API is to use html instead. You can get up to 24 hours worth of tweets if you grab http://twitter.com/user/with_friends (Basic HTTP Auth works, to see friends who's tweets aren't public). You'd have to parse it all out (not as easy or clean as xml/rss/etc.) and watch for the "next" links and get/parse the subsequent pages, but at least the info is available...<br><br>I tried page=2 on various calls in the official API but it doesn't work anywhere.<br><br>If this seems like the way to go, I could write a fairly quick parser in perl that would take the html and output something a little more manageable. Perhaps with a date param so we don't have to go back more pages than we have to.<br><br>VernVernnoreply@blogger.comtag:blogger.com,1999:blog-3715000527290050953.post-79715921146684976132007-04-10T16:22:55.000-07:002007-04-10T16:22:55.000-07:00I just got pointed to this post by Froosh on Twitt...I just got pointed to this post by Froosh on Twitter. I am the author of TwitBox a Windows client for Twitter and I am currently working on new features that I can pull from the API - like search for user - to providing enhancements within TwitBox itself to make using Twitter a little more personable.<br><br>One of the things I will be working on which may not meet your needs for "attention" it might help. Basically it will allow you to define color codes for tweet priorities.<br><br>In the meantime I'll keep an eye on this comment thread incase any suggestions come up that I think I can add.Steven Hodsonhttp://www.winextra.comnoreply@blogger.com