- How the text is split: json value.
- How the chunk size is measured: by number of characters.
Basic usage
Specifymax_chunk_size
to constrain chunk sizes:
.split_json
method:
.create_documents
method:
.split_text
to obtain string content directly:
How to manage chunk sizes from list content
Note that one of the chunks in this example is larger than the specifiedmax_chunk_size
of 300. Reviewing one of these chunks that was bigger we see there is a list object there:
convert_lists=True
to preprocess the json, converting list content to dicts with index:item
as key:val
pairs: