rajdavies: hey I was wrong about that xindice stuff. It seems decent.. I had made a small mistake in my test case.

it seemed like the xindice btree was faster had a more consistent insert/delete rate.

chirino_m: nice!

how big a test case ?

how about the search times ?

could you check the HashIndexBenchmark test.. make sure it's configured right / optimized right

will do

the work load was 1 thread creating records. and 1 thread looking for a record and then deleting the record.

you will find a similar test in https://svn.apache.org/repos/asf/activemq/sandbox/xindice-stripped

BTW their Hash impl also seemed better.

perhaps I had the kaha one mis configured.

yep - 12 bins is a little harsh

can you just use the default ?

sure.

I copied that from the other hash test..

woot ok that looks much better

ok.. it's kicking ass now.

cool!

rajdavies: so you think it's worth doing the double buffering idea?

yeah! not sure how to do it though

I think the main thing we have to do is delay writing the pages to disk..

and alocated about 100 pages at the front of the file for the double buffer.

ok!

ok so even if we avoid the partial page write problem.. we still have the missed updates problem. Which we can redo, using the log

but it would be nice if the index could tell me where in the log to start from.

yes - agreed - there needs to be more info in the kaha indexes

Also - a HashIndex - points to another index which points to the real data - so theirs a lot of room for optimization

there's - sry jstrachan for my bad english!

:)

so is the index only getting updated by the journal now?

in amqstore - yes

cool so in every page we could also record what the journal location was for that update..

yes

and then we can just peak that the double buffer to see what was the last page updated. and get the last journal location.

yes

does each page also store it's id/offset

for the double buffer bit I think I need that.

its id is its offset

but is it stored in the page?

not at the moment

I may need to add that.. just so when I read the page from the double buffer, I know where it's real location should be.

also the next tricky bit is how do I now if the page in the double buffer had a partial page write or if the actual page had the partial page write?

s/now/know

I think we should do this as part of moving kaha into its own separate project

sure.

I get confused by this - is AMQDB the same thing as kaha - or is that considered something else?

is kaha the name for all the custom db stuff? or are there still 2 different beasties?

kaha is different - its used to store the indexes for AMQDB - but is a embeddable db in its own right

ah ok - so its just kaha that's be separate project right? the btree/db thingy

I guess AMQDB is AMQ specific stuff right?

yes - AMQDB is AMQ specific

messages + queue indices / durable topic subscriber stuff etc right?

yep

rajdavies: so lets spin the project then..

where should we put it?

sub-project of AMQ first?

yeah

in sandbox until amq starts using it?

its tempting to pop it straight at activemq/kaha/trunk etc?

I guess we should vote first :)

rdavies: welcome back

I be in favor of starting either a new package for kaha - and fix it there

yeah - sry laptop frooze

rdavies: so where in svn do you want to put it?

ttps://svn.apache.org/repos/asf/activemq/kaha or ttps://svn.apache.org/repos/asf/activemq/sandbox/kaha

ideally a new sub project

that might require a vote and stuff

yeah!

lets kick off a vote

how about we sping up in the sandbox for now.. and start discussing it on dev.. and agree to make it a sub project

that works ok too

so kaha or kahadb ...

adding a db might hint folks to what it is.

its going to take some work to do properly

yeah

yeah - kahadb

you wana do the initial spike.. or should I?

should we change the package to org.apache.kahadb too?

yes

could be a good time to come up with a better name ;)

tired of kaha?

sonicdb ... lol

rdavies: you wana do the initial spike.. or should I?

chirino_m: I'm on vacation - you can ;)

rdavies: oh did not know!

hope your enjoying it.!

still packing - so glad to be out of the way!

in the spirit of berkeleydb, maybe it should be dublindb, londondb or tampadb?

I guess LeightonBuzzardDB is a little long ?

;)

lbdb

like it!

already exists though

best stick with kaha - more descriptive

k

yeah

chirino_m: i wiped out my m2 repo and i have a missing artifact: protobuf-java-2.0.0beta do you know where it's available from ?

its not available yet - but the activemq-protocol-buffer shouldn't be included in the build yet?

I built it locally as it needed a little patch

jstrachan: perahps move it to sandbox?

yeah

I was gonna make the dependency available in some repo

I guess its a big undertaking switching out openwire

I guess we could move it into the amq 6 branch :)

jstrachan: yeah

it'd be nice if we could refactor the broker soon to use some kinda abstract-ish interfaces that we could implement on top of current openwire or something else (e.g. protocol buffer) or whatever - to loosen the dependency between the current openwire command structures and the functionality of the broker

jstrachan: +1

jstrachan: would require some serious surgery tho

yeah

I don't think even IDEA could handle that refactor :)

:)

well https://svn.apache.org/repos/asf/activemq/sandbox/kahadb is in.

coolio

its mostly all those broker/region/subscription classes

be nice to decouple a bit from the underlying wire protocol if we could

jstrachan: yeah

too scared to touch the code much though :)

me too!

might be better waiting for amq6 :)

jstrachan: if you want to experiment in a smaller broker..

we have a little experimental one.

that easier to grok/test ideas on.

feel free to test the idea out on https://svn.apache.org/repos/asf/activemq/sandbox/activemq-router

coolio

btw that ZK code is hard to hack without chaning lots of stuff.

not alot of interfaces or use of DI

jstrachan: commenting out activemq-protocol-buffer from the build worked

coolio

shouldn't i ci this? other users will certainly hit this too

until protobuf-java becomes available

yeah, commit it

I sent a note out to the ZK folks asking if they wouldn't mind making the project more DI.

done

is anyone familiar with activemq-admin stop?