[Boost-users] [MultiIndex] Prevent automatic re-indexing after "modify"?

[Deniz Bahadir]

Hi guys,
Hi Joaquín,

I have a question concerning the automatic re-indexing of
Boost.MultiIndex indexes.

Is it somehow possible to modify a member of the element-type that is
used as key for (another) index without triggering automatic
re-indexing? (Of course, I want to be able to trigger manual re-indexing
later.)
Or is it at least possible to adjust the code so that this can be achieved?


To understand my question, here is my use-case and why I need it.

I have a boost::multi_index_container with multiple indexes (one
hashed_unique and four hashed_non_unique).
I am accessing a range of elements through a hashed_non_unique index
'A'. Then I am iterating through these elements and want to modify the
value of a member 'M' of each element. However, that member is used when
calculating the key for another hashed_non_unique index 'B'.
Therefore, I must use the "modify" function of index 'A'.

The problem is now that it takes forever to modify the entire range. (We
are talking about 100 thousands if not millions of entries that need to
be modified.)
Probably because re-indexing is triggered after each call to "modify".
However, in my case it should be perfectly fine to postpone the
re-indexing after I modified the entire range, because I am not going to
access any element through index 'B' until I finished modifying the
entire range.


As a workaround I removed index 'B' from the container and declared the
element-member 'M' as mutable. This allows me to directly change the
value of 'M'.
The speedup is astonishing. It now finishes almost in no time.

But this workaround is not really feasible as it requires reimplementing
index 'B' by somehow tracking the information outside of the
multi_index_container.


Therefore, my question really is:

Is it possible to provide an "unsafe_modify" function, which does the
same as "modify" but does not trigger automatic re-indexing, and an
"refresh_index" function, which (manually) triggers the re-indexing?


Thanks,
Deniz

[Leon Mlakar]

Post by Deniz Bahadir
Is it possible to provide an "unsafe_modify" function, which does the
same as "modify" but does not trigger automatic re-indexing, and an
"refresh_index" function, which (manually) triggers the re-indexing?

I second this idea. Hit the same issue not so long ago. Or event better,
something like modify_range(<first>, <last>, <modifier>) that would
automatically re-index after modifying entire range thus avoiding the
danger of user forgetting to re-index.

Best regards,
Leon

[Dominique Devienne]

Post by Leon Mlakar

Post by Deniz Bahadir
Is it possible to provide an "unsafe_modify" function, which does the
same as "modify" but does not trigger automatic re-indexing, and an
"refresh_index" function, which (manually) triggers the re-indexing?

I second this idea. Hit the same issue not so long ago. Or event better,
something like modify_range(<first>, <last>, <modifier>) that would
automatically re-index after modifying entire range thus avoiding the
danger of user forgetting to re-index.

If it takes <first>, <last>, it's not a range :) Joking apart, better to
accept anything that range-for can accept, and use Boost.Range to adapt a
pair of iterators into an actual range.

Yes, a modify_range() or bulk_modify() or insert_better_name() would be
great.
But then you don't know which index needs to be rebuilt, so they must all
be rebuilt.
So a new API to do this could additionally explicitly spell out which ones
perhaps.
Although then nothing prevents other keys to be modified, breaking BMI
invariants.
But I for one would be willing to accept the risk for the performance
benefits. --DD

PS: The same applies to the current "scalar" modify I guess, which must
update all indices too.

[Leon Mlakar]

Post by Deniz Bahadir
Is it possible to provide an "unsafe_modify" function, which
does the same as "modify" but does not trigger automatic
re-indexing, and an "refresh_index" function, which (manually)
triggers the re-indexing?
I second this idea. Hit the same issue not so long ago. Or event
better, something like modify_range(<first>, <last>, <modifier>)
that would automatically re-index after modifying entire range
thus avoiding the danger of user forgetting to re-index.
If it takes <first>, <last>, it's not a range :) Joking apart, better
to accept anything that range-for can accept, and use Boost.Range to
adapt a pair of iterators into an actual range.

Well, possibly ... I was sort of trying to model on equal_range search,
which returns pair of first, last iterators... and since it's likely it
would be used on the results of equal_range search, passing them to
modify_range (either as pair or as separate parameters) would be fit.

Post by Deniz Bahadir
Yes, a modify_range() or bulk_modify() or insert_better_name() would
be great.
But then you don't know which index needs to be rebuilt, so they must
all be rebuilt.
So a new API to do this could additionally explicitly spell out which
ones perhaps.
Although then nothing prevents other keys to be modified, breaking BMI
invariants.

I'm not sure I understand this - afaik you can already modify other
key(s) via index::modify so it has to adjust/rebuild all indices.

Cheers,
Leon

[Deniz Bahadir]

Hi Joaquín,

I just wanted to bring this topic up again and ask you if you can give
me some hints on where I might have to look to possibly modify/extend

Post by Deniz Bahadir
Hi guys,
Hi Joaquín,
I have a question concerning the automatic re-indexing of
Boost.MultiIndex indexes.
Is it somehow possible to modify a member of the element-type that is
used as key for (another) index without triggering automatic
re-indexing? (Of course, I want to be able to trigger manual re-indexing
later.)
Or is it at least possible to adjust the code so that this can be achieved?
To understand my question, here is my use-case and why I need it.
I have a boost::multi_index_container with multiple indexes (one
hashed_unique and four hashed_non_unique).
I am accessing a range of elements through a hashed_non_unique index
'A'. Then I am iterating through these elements and want to modify the
value of a member 'M' of each element. However, that member is used when
calculating the key for another hashed_non_unique index 'B'.
Therefore, I must use the "modify" function of index 'A'.
The problem is now that it takes forever to modify the entire range. (We
are talking about 100 thousands if not millions of entries that need to
be modified.)
Probably because re-indexing is triggered after each call to "modify".
However, in my case it should be perfectly fine to postpone the
re-indexing after I modified the entire range, because I am not going to
access any element through index 'B' until I finished modifying the
entire range.
As a workaround I removed index 'B' from the container and declared the
element-member 'M' as mutable. This allows me to directly change the
value of 'M'.
The speedup is astonishing. It now finishes almost in no time.
But this workaround is not really feasible as it requires reimplementing
index 'B' by somehow tracking the information outside of the
multi_index_container.
Is it possible to provide an "unsafe_modify" function, which does the
same as "modify" but does not trigger automatic re-indexing, and an
"refresh_index" function, which (manually) triggers the re-indexing?
Thanks,
Deniz

Thanks again,
Deniz
--
BENOCS GmbH
Dipl.-Inform. Deniz Bahadir
Winterfeldtstr. 21
10781 Berlin
Germany
Phone: +49 - 30 / 577 0004-22
Email: ***@benocs.com
www.benocs.com

Advisory Board (Chairman): Stephan Schröder
Board of Management: Dr.-Ing. Oliver Holschke, Dr.-Ing. Ingmar Poese
Commercial Register: Amtsgericht Bonn HRB 19378