Differences
This shows you the differences between two versions of the page.
|
projs:clans:docs:crawlingretweet [2014/01/26 20:29] yangjunfeng0317 created |
projs:clans:docs:crawlingretweet [2014/02/04 18:32] (current) yangjunfeng0317 |
||
|---|---|---|---|
| Line 8: | Line 8: | ||
| ===== Output ===== | ===== Output ===== | ||
| - | None | + | ^ Parameters ^ Type ^ Description ^ |
| + | | status | string | show the crawler running status | | ||
| ===== Implementation ===== | ===== Implementation ===== | ||
| - masterStart(). Create multiple processes to begin crawling data. | - masterStart(). Create multiple processes to begin crawling data. | ||
| - wapLogIn(). Log in sina Account. | - wapLogIn(). Log in sina Account. | ||
| - | - weiBoWapSearch(person_name, pid). Use person name and person id to search person related weibo | + | - weiBoWapSearch(searchStr, Sid). Use searchStr(person name or company name) and search id(person id or company id) to search related weibo |
| * extractTopic(person_name or company_name, person_id or company_id). Extract weibo text and insert to database. | * extractTopic(person_name or company_name, person_id or company_id). Extract weibo text and insert to database. | ||
| * getRetweet(retweet_url, weibo_id). Extract retweet of original weibo text and insert to database. | * getRetweet(retweet_url, weibo_id). Extract retweet of original weibo text and insert to database. | ||
