Skip to main content

Home/ Coders/ Group items tagged scraping

Rss Feed Group items tagged

Kingdon Barrett

Radiant CMS: Why Radiant? - 0 views

    • Kingdon Barrett
       
      I was hoping to provide a list of my blog posts from RSS feeds, as well as links to all of the articles that I have commented on with my friends' blogs. I could list all of my friends' blogs and scrape them periodically, but I was hoping for a solution that uses trackbacks, or whatever pingback-style facilities are available on my friends' blogs. Granted this type of support will vary from site to site, but it should be possible to build a complete publishing record automatically, with the standards we have available today. I posted a comment on otierney.net (my friend Tristan is up on all the latest web standards) and I noticed that his blogger is collecting URLs along with emails. I filled in the link to my Radiant CMS on nerdland.org but I'm afraid my blogger is not going to do anything to collect these pingbacks, when they are fired. Can anyone point out an example code segment or project that highlights this type of behavior?
Joel Bennett

Deriving from WebClient to handle persisting cookies for login - 0 views

  • Then just use the ExWebClient class to make your requests;
  • Public Class CookieWebClient : Inherits WebClient ' overridden to add cookie headers to http requests. Protected Overrides Function GetWebRequest(ByVal address As System.Uri) As System.Net.WebRequest Dim request As WebRequest = MyBase.GetWebRequest(address) If TypeOf request Is HttpWebRequest Then DirectCast(request, HttpWebRequest).CookieContainer = _cookies End If Return request End Function ' overridden to save cookies to the container for http requests. Protected Overrides Function GetWebResponse(ByVal request As System.Net.WebRequest) As System.Net.WebResponse Dim response As WebResponse = MyBase.GetWebResponse(request) If TypeOf response Is HttpWebResponse Then _cookies.Add(response.ResponseUri, DirectCast(response, HttpWebResponse).Cookies) End If Return response End Function ' overridden to save cookies to the container for async http requests. Protected Overrides Function GetWebResponse(ByVal request As System.Net.WebRequest, ByVal result As System.IAsyncResult) As System.Net.WebResponse Dim response As WebResponse = MyBase.GetWebResponse(request, result) If TypeOf response Is HttpWebResponse Then _cookies.Add(response.ResponseUri, DirectCast(response, HttpWebResponse).Cookies) End If Return response End Function Private Shared _cookies As CookieContainer = New CookieContainer End Class
jo_kruger

An Introduction to Regular Expressions in RL3 - 0 views

  •  
    Regular expressions are extremely handy and powerful tool. In RL3 regex are also human-friendly and easy to write.
1 - 8 of 8
Showing 20 items per page