Group items tagged

.twitter -rss [enter search term(s)]

.twitter -rss-u [enter Twitter user]

.twitter -rss-f [enter Twitter user]

Feeds HTML Parser for Node Creation

I currently have a Drupal site set up and am using the Feeds module (http://drupal.org/project/feeds) to create CCK nodes from a few RSS feeds. This is a standard use case of that module, at least as far as I know.

The need has arisen for the ability to also create content from non-RSS/non-XML sources. I need a new Parser created for the Feeds module that would allow for one to populate CCK fields based on the parsing of raw HTML content. My first thought is that the user should be allowed to define a regular expression for each field, with the field then being populated by the output of the regular expression applied to the raw HTML content. However, I am open to suggestions on different solutions which might be easier for the developer.

Ive written a small js application that gets html content as a string (this only has to work in mozilla)

req = new XMLHttpRequest(); req.open('GET', URI, true); req.onreadystatechange = function (aEvt) { if (req.readyState == 4) { if(req.status == 200){ var myTxt = req.responseText;

This works fine - myTxt contains the html code as a string. However I want to be able to parse the code using the DOM... Is there a way to create an HTML DOM in Javascript easily?

Parsing HTML in Python

Up until now, I've avoided doing any HTMl parsing in my RSS reader FeedMe.

from HTMLParser import HTMLParser class MyFancyHTMLParser(HTMLParser): def fetch_url(self, url) : request = urllib2.Request(url) response = urllib2.urlopen(request) link = response.geturl() html = response.read() response.close() self.feed(html) # feed() starts the HTMLParser parsing def handle_starttag(self, tag, attrs): if tag == 'img' : # attrs is a list of tuples, (attribute, value) srcindex = self.has_attr('src', attrs) if srcindex < 0 : return # img with no src tag? skip it src = attrs[srcindex][1] # Make relative URLs absolute src = self.make_absolute(src) attrs[srcindex] = (attrs[srcindex][0], src) print '<' + tag for attr in attrs : print ' ' + attr[0] if len(attr) > 1 and type(attr[1]) == 'str' : # make sure attr[1] doesn't have any embedded double-quotes val = attr[1].replace('"', '\"') print '="' + val + '"') print '>' def handle_endtag(self, tag): self.outfile.write('</' + tag.encode(self.encoding) + '>\n'