MySQL optimization on join tables with range criteria -

-

i going join 2 tables using single position in 1 table range (represented 2 columns) in table.

however, performance slow, 20 mins. have tried adding index on table or changing query. performance still poor.

so, asking optimization of joining speed.

the following query mysql.

mysql> select `invar`.chrom, `invar`.pos, `openchrom_k562`.score     -> `invar`     -> left join `openchrom_k562`     -> on (     -> `invar`.chrom=`openchrom_k562`.chrom ,     -> `invar`.pos between `openchrom_k562`.chromstart , `openchrom_k562`.chromend     -> );

invar , openchrom_k562 tables used.

invar stores single position in each row.

openchrom_k562 stores range information indicated chromstart , chromend.

invar contains 57902 rows , openchrom_k562 has 137373 rows respectively.

fields on tables.

mysql> describe invar; +-------+-------------+------+-----+---------+-------+ | field | type        | null | key | default | | +-------+-------------+------+-----+---------+-------+ | chrom | varchar(31) | no   | pri | null    |       | | pos   | int(10)     | no   | pri | null    |       | +-------+-------------+------+-----+---------+-------+  mysql> describe openchrom_k562; +------------+-------------+------+-----+---------+-------+ | field      | type        | null | key | default | | +------------+-------------+------+-----+---------+-------+ | chrom      | varchar(31) | no   | mul | null    |       | | chromstart | int(10)     | no   | mul | null    |       | | chromend   | int(10)     | no   |     | null    |       | | score      | int(10)     | no   |     | null    |       | +------------+-------------+------+-----+---------+-------+

index built in tables

mysql> show index invar; +-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ | table | non_unique | key_name | seq_in_index | column_name | collation | cardinality | sub_part | packed | null | index_type | comment | +-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ | invar |          0 | primary  |            1 | chrom       |         |        null |     null | null   |      | btree      |         | | invar |          0 | primary  |            2 | pos         |         |       57902 |     null | null   |      | btree      |         | +-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+  mysql> show index openchrom_k562; +----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ | table          | non_unique | key_name    | seq_in_index | column_name | collation | cardinality | sub_part | packed | null | index_type | comment | +----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ | openchrom_k562 |          1 | start_end   |            1 | chromstart  |         |      137373 |     null | null   |      | btree      |         | | openchrom_k562 |          1 | start_end   |            2 | chromend    |         |      137373 |     null | null   |      | btree      |         | | openchrom_k562 |          1 | chrom_only  |            1 | chrom       |         |          22 |     null | null   |      | btree      |         | | openchrom_k562 |          1 | chrom_start |            1 | chrom       |         |          22 |     null | null   |      | btree      |         | | openchrom_k562 |          1 | chrom_start |            2 | chromstart  |         |      137373 |     null | null   |      | btree      |         | | openchrom_k562 |          1 | chrom_end   |            1 | chrom       |         |          22 |     null | null   |      | btree      |         | | openchrom_k562 |          1 | chrom_end   |            2 | chromend    |         |      137373 |     null | null   |      | btree      |         | +----------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+

execution plan on mysql

mysql> explain select `invar`.chrom, `invar`.pos, score  `invar`  left join `openchrom_k562`  on ( invar.chrom=openchrom_k562.chrom ,  `invar`.pos between chromstart , chromend ); +----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+ | id | select_type | table          | type  | possible_keys                              | key        | key_len | ref             | rows  |       | +----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+ |  1 | simple      | invar          | index | null                                       | primary    | 37      | null            | 57902 | using index | |  1 | simple      | openchrom_k562 | ref   | start_end,chrom_only,chrom_start,chrom_end | chrom_only | 33      | tmp.invar.chrom |  5973 |             | +----+-------------+----------------+-------+--------------------------------------------+------------+---------+-----------------+-------+-------------+

it seems optimizes looking chrom in 2 tables. brute-force comparing in tables.

is there ways further optimization indexing on position?

(it first time posting question, sorry poor posting quality.)

chrom_only bad index selection join have chrom 22 values.

if have interpreted right query should faster if using start_end

select `invar`.chrom, `invar`.pos, `openchrom_k562`.score `invar` left join `openchrom_k562` use index (`start_end`) on ( `invar`.chrom=`openchrom_k562`.chrom , `invar`.pos between `openchrom_k562`.chromstart , `openchrom_k562`.chromend )

Comments

Post a Comment

Popular posts from this blog

django - How can I change user group without delete record -

-
on website user have 1 group. , user can change group. it's made user.groups.clear() and user.groups.add(new_group) but it's not efficient, because there 2 sql query: delete, insert. how can change group update query? user , group related each other using manytomanyfield . means intersection table exists relating both entities, , if don't specify model map (using through attribute) django creates 1 you. looking @ sources django.contrib.auth.models see that's case. fortunatly, can access intermediary model using through attribute of manager (in case, user.groups.through ). can use regular model. example: >>> alice = user.objects.create_user('alice', 'alice@example.com', 'alicepw') >>> employee = group.objects.create(name='employee') >>> manager = group.objects.create(name='manager') >>> alice.groups.add(employee) >>> alice.groups.all() [<group: employee...
Read more

java - Need to add SOAP security token -

-
i need set custom soap header attribute on jax-ws generated webservice client. case webservice calls must go through proxy server requiring specific token (recieved web request header) present in soap request header. e.g.: 1 carserviceservice service = null; 2 service = new carserviceservice(new url(url), new qname(qname); 3 carserviceendpoint port = service.getcarserviceport(); it seems in line 3 wsdl retrieved , call fails due missing security token. 1 point direction on how done? a detailed example has been mentioned here: creating , deploying jax-ws web service on tomcat 6 this article shows how create , use security token .
Read more

java - EclipseLink JPA Object is not a known entity type -

-
i have strange problem eclipselink , object want persist. have 1 object (keypointlistimpl) stores object keypointimpl in list. persisting keypointimpl objects works great if try persist keypointlistimpl object java.lang.illegalargumentexception says object keypointimpl isn't known entity type. here keypointimpl code: @entity @table(name="keypoints") public class keypointimpl implements keypoint { @id @generatedvalue(strategy = generationtype.identity) private int id; @enumerated(enumtype.string) private detectortype keypointtype; private float x; private float y; private float size; private float angle; private float response; private int octave; private int classid; ... } here keypointlistimpl code: @entity @table(name="keypointlists") public class keypointlistimpl implements keypointlist { @id @generatedvalue(strategy = generationtype.identity) private int id; @onetoone(cascade={ca...
Read more