machine learning - Adding new terms to a bag-of-words model -

-

i'm using k-means clustering group set of news items. i'm using bag-of-words model represent documents, more specifically, each document represented term frequency vector.

my question: how can add new documents without having recalculate term frequency vectors (seen vocabulary containing terms docs change)?

the easy solution use vocabulary documents you've seen, ignoring new terms; customary in document classification.

another solution, that's become popular in recent years, ditch vocabulary altogether , use feature hashing.

a third possibility reserve space in feature vectors future terms. e.g., let's you're vectorizing bunch of documents vocabulary of size n, turn them vectors of size n + k final k set zero, can add k terms vocabulary later on.

(what not solution compute dot products, means, etc. on hash tables directly. flexible approach, it's very slow.)


Comments

Post a Comment

Popular posts from this blog

How do you change vbulletin's home page to the forums instead of the default (CMS or Activity Stream)? -

-
i have created website using vbulletin have installed cms , blog modules in in admin default setting forums when enter homepage url redirecting me http://www.demo.com/content/ content cms page please help for vbulletin versions < 4.2. need modify forum index page located in top level of vbulletin directory. based on url in question it's file: http://www.demo.com/index.php here contents of file, can choose whether cms or forum default home page. /** * can choose default script here. uncomment appropriate line * set default script. note: uncomment 1 of these, must * add // comment out script(s) not want use * default script. * * can choose default script if not plan move * file root of website. */ /** * use cms default script: */ require('content.php'); /** * use forum default script: */ // require('forum.php'); for vbulletin versions >= 4.2. there's new "navigation manager". you'll find control settin...
Read more

c++ - Troubles when compiling phash program -

-
here (probably) simple problem: i'm trying use perceptual hashing library phash ubuntu 11.10. had ffmpeg installed, way i've done: sudo apt-get install libphash0 sudo apt-get install libphash0-dev then tried compile program: #include <iostream> #include <phash.h> using namespace std; int main() { ulong64 myhash=0; ph_dct_imagehash("test.jpg", myhash); cout<<myhash<<endl; } when compiling, prints out: undefined reference `ph_dct_imagehash' any suggestion? should do? in advance! matteo monti you forgot link phash library, linker cannot find function. try adding -l phash gcc command line (or update makefile). if doesn't work, maybe you'll need specify library path (the location of *.a file) using -l "/usr/lib/"
Read more