dossier/store/store.py
def put(self, items, indexes=True):
        '''Add feature collections to the store.

        Given an iterable of tuples of the form
        ``(content_id, feature collection)``, add each to the store
        and overwrite any that already exist.

        This method optionally accepts a keyword argument `indexes`,
        which by default is set to ``True``. When it is ``True``,
        it will *create* new indexes for each content object for all
        indexes defined on this store.

        Note that this will not update existing indexes. (There is
        currently no way to do this without running some sort of
        garbage collection process.)

        :param iterable items: iterable of
                               ``(content_id, FeatureCollection)``.
        :type fc: :class:`dossier.fc.FeatureCollection`
        '''
        # So why accept an iterable? Ideally, some day, `kvlayer.put` would
        # accept an iterable, so we should too.
        #
        # But we have to transform it to a list in order to update indexes
        # anyway. Well, if we don't have to update indexes, then we can avoid
        # loading everything into memory, which seems like an optimization
        # worth having even if it's only some of the time.
        #
        # N.B. If you're thinking, "Just use itertools.tee", then you should
        # heed this warning from Python docs: "This itertool may require
        # significant auxiliary storage (depending on how much temporary data
        # needs to be stored). In general, if one iterator uses most or all of
        # the data before another iterator starts, it is faster to use list()
        # instead of tee()."
        #
        # i.e., `tee` has to store everything into memory because `kvlayer`
        # will exhaust the first iterator before indexes get updated.
        items = list(items)
        self.kvl.put(self.TABLE,
                     *imap(lambda (cid, fc): ((cid,), fc_dumps(fc)), items))
        if indexes:
            for idx_name in self._indexes:
                self._index_put(idx_name, *items)
def put(self, items, indexes=True):
        '''Add feature collections to the store.

        Given an iterable of tuples of the form
        ``(content_id, feature collection)``, add each to the store
        and overwrite any that already exist.

        This method optionally accepts a keyword argument `indexes`,
        which by default is set to ``True``. When it is ``True``,
        it will *create* new indexes for each content object for all
        indexes defined on this store.

        Note that this will not update existing indexes. (There is
        currently no way to do this without running some sort of
        garbage collection process.)

        :param iterable items: iterable of
                               ``(content_id, FeatureCollection)``.
        :type fc: :class:`dossier.fc.FeatureCollection`
        '''
        # So why accept an iterable? Ideally, some day, `kvlayer.put` would
        # accept an iterable, so we should too.
        #
        # But we have to transform it to a list in order to update indexes
        # anyway. Well, if we don't have to update indexes, then we can avoid
        # loading everything into memory, which seems like an optimization
        # worth having even if it's only some of the time.
        #
        # N.B. If you're thinking, "Just use itertools.tee", then you should
        # heed this warning from Python docs: "This itertool may require
        # significant auxiliary storage (depending on how much temporary data
        # needs to be stored). In general, if one iterator uses most or all of
        # the data before another iterator starts, it is faster to use list()
        # instead of tee()."
        #
        # i.e., `tee` has to store everything into memory because `kvlayer`
        # will exhaust the first iterator before indexes get updated.
        items = list(items)
        self.kvl.put(self.TABLE,
                     *imap(lambda (cid, fc): ((cid,), fc_dumps(fc)), items))
        if indexes:
            for idx_name in self._indexes:
                self._index_put(idx_name, *items)
Add feature collections to the store.

        Given an iterable of tuples of the form
        ``(content_id, feature collection)``, add each to the store
        and overwrite any that already exist.

        This method optionally accepts a keyword argument `indexes`,
        which by default is set to ``True``. When it is ``True``,
        it will *create* new indexes for each content object for all
        indexes defined on this store.

        Note that this will not update existing indexes. (There is
        currently no way to do this without running some sort of
        garbage collection process.)

        :param iterable items: iterable of
                               ``(content_id, FeatureCollection)``.
        :type fc: :class:`dossier.fc.FeatureCollection`