Sujit Pal
sujitpal.blogspot.com · Personal blog · on HN since 2009
Sujit Pal writes about search, text mining and machine learning, with Elasticsearch, NLTK, Python and Hadoop examples.
What HN saysHN links it for the Elasticsearch image search post, and the author joined comments to discuss it and a follow-up comparison.
15HN people shared it
4linked it in comments
11stories submitted
149story points
8 years agolast shared
2009people per year2026
Most-shared posts
- 1Salmon Run: More Java Actor Frameworks Compared2 people · 3 points · first shared 2009 · discussion on HN
- 2Elasticsearch Based Image Search Using RGB Signatures1 people · 126 points · first shared 2016 · 19 comments on HN
- 3Hierarchical Agglomerative Clustering with Hadoop1 people · 5 points · first shared 2009 · 1 comments on HN
- 4Assembling a Python Machine Learning Toolkit1 people · 3 points · first shared 2013 · discussion on HN
- 5Keyword deduplication using the Python dedupe library1 people · 2 points · first shared 2018 · discussion on HN
- 6The Benefits of Attention for Document Classification1 people · 2 points · first shared 2017 · discussion on HN
- 7Document Similarity Using Various Text Vectorizing Strategies1 people · 2 points · first shared 2016 · discussion on HN
- 8Finding Significant Phrases in Tweets with NLTK1 people · 2 points · first shared 2013 · discussion on HN
- 9Categorizing Medical Transcripts Using DMOZ and Word2Vec1 people · 2 points · first shared 2015 · discussion on HN
- 10Scalding For The Impatient1 people · 1 points · first shared 2013 · discussion on HN
- 11The Wikipedia Bob Alice HMM Example Using Scikit-Learn1 people · 1 points · first shared 2015 · discussion on HN
- 12Comparison of image search performance1 people · first shared 2016
- 13Summarization with lucene1 people · first shared 2012
- 14Elasticsearch based image search using1 people · first shared 2016
What HN people wrote when they linked it
softwaredoug · 2016-06-12
(Author here) Case in point where Elasticsearch shines is one of yesterday's hacker news articles
http://sujitpal.blogspot.com/2016/05/elasticsearch-based-image-search-…
Here Elasticsearch is used as a framework where "search" really means some kind of distributed feature similarity. But even text search where your incorporating a lot of data science, external signals, or semantic awareness can fall more into the category of distributed feature similarity.
Algolias strength is in a simpler path to straightforward and easily understood search. I'm not sure it's the path to amazing and deeply customized search (or search-driven features). Algolia gets far more right out of the box at a configuration level many need when they can't put a team around building an amazing Solr/ES experience.
My hope is Solr/ES can learn a thing or two in the ease of use dept with relevance!
sujitpal · 2016-06-19
Hello, author of the post here. Thanks for all your valuable comments. I noticed that there was some talk about neural network driven models. In my follow up post to this one:
http://sujitpal.blogspot.com/2016/06/comparison-of-image-search-perfor…
I describe how I use transfer learning to generate image vectors for my butterfly images from the Caffe reference model trained on ImageNet. There are some other approaches too, but probably not as interesting to this audience.
@GrantS - thanks for the intro and the links, and for pointing out that I was using BoVW incorrectly. Not an image person, trying to get into it from a search/NLP background.
@infinitone and @deckar01 - I measured the goodness of search by checking how many times the query butterfly was the #1 result. I agree its pretty hard to distinguish butterflies from one another, but I needed something to work with, and that was as good as any. I did not want one that crosses subject areas, such as flowers and cars, so a search for a red flower might bring back a red car.
@chrischen - if you got rid of the bucketing portion in my pipeline, I think you might achieve what you are after. But if you are only looking for exact match on color, then hashing might be cheaper.
@rcarmo - thanks for the reference to pHash, I think it might work better in near-duplicate detection kind of cases. Will check it out.
lucifer · 2010-02-25
"Asynchronious message passing is old and great idea. Instead of synchronizing objects and dealing with deadlocks we try to isolate objects and let them exchange messages. Erlang proved it to be extremly powerful tool applicable to wide range of highly scalable mission critical applications.
In JVM landscape big interest to this approach started after introducing Scala actors. "
That statement is so very poorly informed! JMS ("surprise"?) has been a sign of "interest" in "Asynchronious message passing" in the "JVM landscape".
I am not up on my Scala, but the author fails to convey (based on what I do know ;) the competence level that would lend weight to his attempt to address (much less support) the topic.
For those interested in other JVM alternatives, Kilim seems quite promising and seems to hold its own quite well in benchmarks against Erlang.
http://www.malhar.net/sriram/kilim/
http://www.youtube.com/watch?v=37NaHRE0Sqw
http://sujitpal.blogspot.com/2009/01/more-java-actor-frameworks-compar… [scroll to end to see the impressive results for Kilim.]
http://osl.cs.uiuc.edu/docs/pppj09/paper.pdf
[edit: added pdf link]
nl · 2012-04-13
Sure:
My algorithm is something I made up, and from memory it works like this:
1) Remove HTML, stem, remove stopwords etc
2) Sort unique words by popularity in the text
3) Split the original text on sentence boundaries.
4) Include each sentence that first mentions the next most popular word, until the summary is the maximum length requested.
http://news.ycombinator.com/item?id=1803020
Googling turns up http://sujitpal.blogspot.com.au/2009/02/summarization-with-lucene.html which compares a few approaches.
Edit: Also http://stackoverflow.com/questions/2829303/given-a-document-select-a-r… which I think is by Dn_Ab who wrote the OP.