Sujit Pal

sujitpal.blogspot.com · Personal blog · on HN since 2009

Sujit Pal writes about search, text mining and machine learning, with Elasticsearch, NLTK, Python and Hadoop examples.

What HN saysHN links it for the Elasticsearch image search post, and the author joined comments to discuss it and a follow-up comparison.
15HN people shared it
4linked it in comments
11stories submitted
149story points
8 years agolast shared
2009people per year2026

Most-shared posts

  1. 1
    Salmon Run: More Java Actor Frameworks Compared
    2 people · 3 points · first shared 2009 · discussion on HN
  2. 2
    Elasticsearch Based Image Search Using RGB Signatures
    1 people · 126 points · first shared 2016 · 19 comments on HN
  3. 3
    Hierarchical Agglomerative Clustering with Hadoop
    1 people · 5 points · first shared 2009 · 1 comments on HN
  4. 4
    Assembling a Python Machine Learning Toolkit
    1 people · 3 points · first shared 2013 · discussion on HN
  5. 5
    Keyword deduplication using the Python dedupe library
    1 people · 2 points · first shared 2018 · discussion on HN
  6. 6
    The Benefits of Attention for Document Classification
    1 people · 2 points · first shared 2017 · discussion on HN
  7. 7
  8. 8
    Finding Significant Phrases in Tweets with NLTK
    1 people · 2 points · first shared 2013 · discussion on HN
  9. 9
    Categorizing Medical Transcripts Using DMOZ and Word2Vec
    1 people · 2 points · first shared 2015 · discussion on HN
  10. 10
    Scalding For The Impatient
    1 people · 1 points · first shared 2013 · discussion on HN
  11. 11
    The Wikipedia Bob Alice HMM Example Using Scikit-Learn
    1 people · 1 points · first shared 2015 · discussion on HN
  12. 12
    Comparison of image search performance
    1 people · first shared 2016
  13. 13
    Summarization with lucene
    1 people · first shared 2012
  14. 14
    Elasticsearch based image search using
    1 people · first shared 2016

What HN people wrote when they linked it

softwaredoug · 2016-06-12
(Author here) Case in point where Elasticsearch shines is one of yesterday's hacker news articles http://sujitpal.blogspot.com/2016/05/elasticsearch-based-image-search-… Here Elasticsearch is used as a framework where "search" really means some kind of distributed feature similarity. But even text search where your incorporating a lot of data science, external signals, or semantic awareness can fall more into the category of distributed feature similarity. Algolias strength is in a simpler path to straightforward and easily understood search. I'm not sure it's the path to amazing and deeply customized search (or search-driven features). Algolia gets far more right out of the box at a configuration level many need when they can't put a team around building an amazing Solr/ES experience. My hope is Solr/ES can learn a thing or two in the ease of use dept with relevance!
sujitpal · 2016-06-19
Hello, author of the post here. Thanks for all your valuable comments. I noticed that there was some talk about neural network driven models. In my follow up post to this one: http://sujitpal.blogspot.com/2016/06/comparison-of-image-search-perfor… I describe how I use transfer learning to generate image vectors for my butterfly images from the Caffe reference model trained on ImageNet. There are some other approaches too, but probably not as interesting to this audience. @GrantS - thanks for the intro and the links, and for pointing out that I was using BoVW incorrectly. Not an image person, trying to get into it from a search/NLP background. @infinitone and @deckar01 - I measured the goodness of search by checking how many times the query butterfly was the #1 result. I agree its pretty hard to distinguish butterflies from one another, but I needed something to work with, and that was as good as any. I did not want one that crosses subject areas, such as flowers and cars, so a search for a red flower might bring back a red car. @chrischen - if you got rid of the bucketing portion in my pipeline, I think you might achieve what you are after. But if you are only looking for exact match on color, then hashing might be cheaper. @rcarmo - thanks for the reference to pHash, I think it might work better in near-duplicate detection kind of cases. Will check it out.
lucifer · 2010-02-25
"Asynchronious message passing is old and great idea. Instead of synchronizing objects and dealing with deadlocks we try to isolate objects and let them exchange messages. Erlang proved it to be extremly powerful tool applicable to wide range of highly scalable mission critical applications. In JVM landscape big interest to this approach started after introducing Scala actors. " That statement is so very poorly informed! JMS ("surprise"?) has been a sign of "interest" in "Asynchronious message passing" in the "JVM landscape". I am not up on my Scala, but the author fails to convey (based on what I do know ;) the competence level that would lend weight to his attempt to address (much less support) the topic. For those interested in other JVM alternatives, Kilim seems quite promising and seems to hold its own quite well in benchmarks against Erlang. http://www.malhar.net/sriram/kilim/ http://www.youtube.com/watch?v=37NaHRE0Sqw http://sujitpal.blogspot.com/2009/01/more-java-actor-frameworks-compar… [scroll to end to see the impressive results for Kilim.] http://osl.cs.uiuc.edu/docs/pppj09/paper.pdf [edit: added pdf link]
Sure: My algorithm is something I made up, and from memory it works like this: 1) Remove HTML, stem, remove stopwords etc 2) Sort unique words by popularity in the text 3) Split the original text on sentence boundaries. 4) Include each sentence that first mentions the next most popular word, until the summary is the maximum length requested. http://news.ycombinator.com/item?id=1803020 Googling turns up http://sujitpal.blogspot.com.au/2009/02/summarization-with-lucene.html which compares a few approaches. Edit: Also http://stackoverflow.com/questions/2829303/given-a-document-select-a-r… which I think is by Dn_Ab who wrote the OP.

Keyboard shortcuts

/Search
j kNext / previous blog
EnterOpen selected blog
vVisit the blog site
fFollow / unfollow
lSwitch grid / list view
dSwitch dark mode
g hHome
g pTop posts
g tTopics
g fFollowing
g lLiked
g yHistory
g qFAQ
EscBack / close