machine learning - Mahout: RowSimilarity vs Clustering -

-

i trying cluster documents using kmeansclustering approach , created clusters. saved cluster id corresponding particular document recommendations. whenever wanted recommend documents similar particular document, query documents in particular cluster , return n random documents cluster. however, returning random document cluster did not seem appropriate , read somewhere should returning documents nearest document in question.

so started searching calculating distance between documents , stumbled upon rowsimilarity approach returns 10 similar documents each document, ordered distance. approach relies on similarity metric loglikelihood etc calculate distance between documents.

now question this. how clustering better/worse rowsimilarity given both approaches use similarity distance metric calculate distance between documents?

what i'm trying achieve i'm trying cluster products on basis of titles , other text properties recommend similar products. appreciated.

clustering not variant of classification or recommendation. different discipline.

when doing cluster analysis, want discover structure in data. then, should analyzing structure found.

now k-means not meant documents. tries find near optimal partitioning of data set k voronoi cells. unless have reason believe voronoi cells partitioning data, algorithm may pretty useless. because returns result not @ indicate result useful.

for documents, euclidean distance (and k-means in fact optimizing euclidean distances) pretty meaningless. vectors sparse, , k-means cluster centers resemble impossible (and insensible) "average documents".

and havn't started on need find appropriate value of k, on mahout implementation being approximation of lloyds k-means approximation, , on. did check cluster sizes? in situations these, k-means produce degenerate results. example, clusters containing 1 or 0 elements, , mega-cluster containing rest. in situation, might in fact returning random documents database...

just because can use not mean helpful. make sure validate individual steps of approach, example if clusters in way useful , sensible!


Comments

Post a Comment

Popular posts from this blog

django - How can I change user group without delete record -

-
on website user have 1 group. , user can change group. it's made user.groups.clear() and user.groups.add(new_group) but it's not efficient, because there 2 sql query: delete, insert. how can change group update query? user , group related each other using manytomanyfield . means intersection table exists relating both entities, , if don't specify model map (using through attribute) django creates 1 you. looking @ sources django.contrib.auth.models see that's case. fortunatly, can access intermediary model using through attribute of manager (in case, user.groups.through ). can use regular model. example: >>> alice = user.objects.create_user('alice', 'alice@example.com', 'alicepw') >>> employee = group.objects.create(name='employee') >>> manager = group.objects.create(name='manager') >>> alice.groups.add(employee) >>> alice.groups.all() [<group: employee...
Read more

java - Need to add SOAP security token -

-
i need set custom soap header attribute on jax-ws generated webservice client. case webservice calls must go through proxy server requiring specific token (recieved web request header) present in soap request header. e.g.: 1 carserviceservice service = null; 2 service = new carserviceservice(new url(url), new qname(qname); 3 carserviceendpoint port = service.getcarserviceport(); it seems in line 3 wsdl retrieved , call fails due missing security token. 1 point direction on how done? a detailed example has been mentioned here: creating , deploying jax-ws web service on tomcat 6 this article shows how create , use security token .
Read more

java - EclipseLink JPA Object is not a known entity type -

-
i have strange problem eclipselink , object want persist. have 1 object (keypointlistimpl) stores object keypointimpl in list. persisting keypointimpl objects works great if try persist keypointlistimpl object java.lang.illegalargumentexception says object keypointimpl isn't known entity type. here keypointimpl code: @entity @table(name="keypoints") public class keypointimpl implements keypoint { @id @generatedvalue(strategy = generationtype.identity) private int id; @enumerated(enumtype.string) private detectortype keypointtype; private float x; private float y; private float size; private float angle; private float response; private int octave; private int classid; ... } here keypointlistimpl code: @entity @table(name="keypointlists") public class keypointlistimpl implements keypointlist { @id @generatedvalue(strategy = generationtype.identity) private int id; @onetoone(cascade={ca...
Read more