Re: [ANN] DefiDive A lightweight crypto price, news and information terminal
@Seal
Can I ask how you're scraping and aggregating the news? I tried long ago to make the same thing for me but ended with horrible results :/ (yahoo pipe was so great for it)
For the social media, I guess you're using API but the websites (with no RSS)?
As for suggesting more sources, there are sooo many. BitcoinMagazine.com bitcoinist.com...
You can even think to play with google alerts
Nice challenge you're taking to compete with others because that's a huge task.
Easy to deal with duplications if Regex is used
Can I ask how you're scraping and aggregating the news? I tried long ago to make the same thing for me but ended with horrible results :/ (yahoo pipe was so great for it)
For the social media, I guess you're using API but the websites (with no RSS)?
As for suggesting more sources, there are sooo many. BitcoinMagazine.com bitcoinist.com...
You can even think to play with google alerts
Nice challenge you're taking to compete with others because that's a huge task.
The news area could need more newspaper sources, not only CoinTelegraph 
(I don't know, btctimes and others, although I know that if they're too many you'll end up with duplications)
(I don't know, btctimes and others, although I know that if they're too many you'll end up with duplications)Easy to deal with duplications if Regex is used
Hey @LeGaulois, its a difficult task scraping together all of these sources.
We use a mix of APIs and RSS feeds although as you've noticed, it is challenging as the data can be quite messy.
Some of the data sources are structured so that we can pull out headings and links but again its not easy to do consistently without it breaking often. Either way we are committed to the challenge because of how much time a properly implemented news feed aggregator can save people.
It is also on our todo list to check out google alerts and news but I'm not sure if we can make that data useful. If there are any developers who want to contribute to what we are doing then let us know.