search - Optimal Solr JVM/Virtual/Physical Memory Configuration -

-

our company has several different ways of getting leads, , several types of leads deal with. there slight differences between each type of lead , of information shared or related 1 or more other lead types. me , team trying build/configure index using solr handles each of these lead types , shared data .. customer data, resort data. etc (around 1.2 million records in all). we're hosting ubuntu server (12g ram, 8 core opteron), running tomcat 6 , solr 3.4.

i'd index add records in live time when customer submits lead-gen form on our website(around 1500-2000 daily), update when employees add or modify data (around 2500-3000 times daily).

in addition need customers on website , employees in house able search data filters, facets, auto-completes, highlighting , stuff 1 has come expect written search.

this setup functioning, hangs updating records both on website , in our internal apps. commits done every 1000 documents or 5 seconds , optimize once daily. optimal jvm, server or solr configurations type of setup? appreciated , can provide information needed willing help.

first, you should not optimize.

there 2 common erros when configuring jvm heap size in solr:

  • giving memory jvm, (the os cache won't able cache disk operations),
  • giving not enough memory jvm (there lot of pressure on garbage collector forced run frequent stop-the-world collections, use jmx monitoring figure out whether full gc triggered).

one other reason why application may hang background merges. lucene based on segments, , whenever number of segments gets higher mergefactor, merge triggered. low value of mergefactor might explain hangs.

you should give more details on current setup can you:

  • jvm size,
  • what collector using (g1, throughput collector, concurrent low pause collector, ...)
  • index size (on disk, not number of documents),
  • mergefactor, rambuffersizemb, ...

Comments

Post a Comment

Popular posts from this blog

django - How can I change user group without delete record -

-
on website user have 1 group. , user can change group. it's made user.groups.clear() and user.groups.add(new_group) but it's not efficient, because there 2 sql query: delete, insert. how can change group update query? user , group related each other using manytomanyfield . means intersection table exists relating both entities, , if don't specify model map (using through attribute) django creates 1 you. looking @ sources django.contrib.auth.models see that's case. fortunatly, can access intermediary model using through attribute of manager (in case, user.groups.through ). can use regular model. example: >>> alice = user.objects.create_user('alice', 'alice@example.com', 'alicepw') >>> employee = group.objects.create(name='employee') >>> manager = group.objects.create(name='manager') >>> alice.groups.add(employee) >>> alice.groups.all() [<group: employee...
Read more

java - Need to add SOAP security token -

-
i need set custom soap header attribute on jax-ws generated webservice client. case webservice calls must go through proxy server requiring specific token (recieved web request header) present in soap request header. e.g.: 1 carserviceservice service = null; 2 service = new carserviceservice(new url(url), new qname(qname); 3 carserviceendpoint port = service.getcarserviceport(); it seems in line 3 wsdl retrieved , call fails due missing security token. 1 point direction on how done? a detailed example has been mentioned here: creating , deploying jax-ws web service on tomcat 6 this article shows how create , use security token .
Read more

java - EclipseLink JPA Object is not a known entity type -

-
i have strange problem eclipselink , object want persist. have 1 object (keypointlistimpl) stores object keypointimpl in list. persisting keypointimpl objects works great if try persist keypointlistimpl object java.lang.illegalargumentexception says object keypointimpl isn't known entity type. here keypointimpl code: @entity @table(name="keypoints") public class keypointimpl implements keypoint { @id @generatedvalue(strategy = generationtype.identity) private int id; @enumerated(enumtype.string) private detectortype keypointtype; private float x; private float y; private float size; private float angle; private float response; private int octave; private int classid; ... } here keypointlistimpl code: @entity @table(name="keypointlists") public class keypointlistimpl implements keypointlist { @id @generatedvalue(strategy = generationtype.identity) private int id; @onetoone(cascade={ca...
Read more