{ "version": "1.0", "source_chunks_path": "01_chunks.json", "strategy": { "front_ratio": 0.15, "middle_positions": [ 0.25, 0.5, 0.75 ], "tail_ratio": 0.1, "high_density_limit": 30, "front_cap": 12, "tail_cap": 8, "middle_radius": 1, "preview_chars": 220 }, "samples": [ { "sample_id": "OS0001", "chunk_id": "C0001", "start_para": 1, "end_para": 17, "sampling_reason": "front_loaded", "char_len": 2127, "text_preview": "CHAPTER I. Down the Rabbit-Hole Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pi...", "chunk_index": 1, "all_reasons": [ "front_loaded", "high_entity_density" ], "density_score": 0.4, "signal_terms": { "chapter_headings": [], "quoted_phrases": [], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 0, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0002", "chunk_id": "C0002", "start_para": 17, "end_para": 29, "sampling_reason": "front_loaded", "char_len": 2173, "text_preview": "what she was coming to, but it was too dark to see anything; then she looked at the sides of the well, and noticed that they were filled with cupboards and book-shelves; here and there she saw maps and pictures hung upon...", "chunk_index": 2, "all_reasons": [ "front_loaded", "middle_stratified", "high_entity_density" ], "density_score": 0.9, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "ORANGE MARMALADE", "Well" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 2, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0003", "chunk_id": "C0003", "start_para": 29, "end_para": 43, "sampling_reason": "middle_stratified", "char_len": 2067, "text_preview": "to ask: perhaps I shall see it written up somewhere.” Down, down, down. There was nothing else to do, so Alice soon began talking again. “Dinah’ll miss me very much to-night, I should think!” (Dinah was the cat.) “I hope...", "chunk_index": 3, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 0.65, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "Do bats eat cats" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0004", "chunk_id": "C0004", "start_para": 43, "end_para": 55, "sampling_reason": "middle_stratified", "char_len": 2125, "text_preview": "Suddenly she came upon a little three-legged table, all made of solid glass; there was nothing on it except a tiny golden key, and Alice’s first thought was that it might belong to one of the doors of the hall; but, alas...", "chunk_index": 4, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 1.15, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "DRINK ME", "Drink me", "No, I’ll look first" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 3, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0005", "chunk_id": "C0005", "start_para": 55, "end_para": 72, "sampling_reason": "middle_stratified", "char_len": 2166, "text_preview": "several nice little histories about children who had got burnt, and eaten up by wild beasts and other unpleasant things, all because they _would_ not remember the simple rules their friends had taught them: such as, that...", "chunk_index": 5, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 0.9, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "poison", "What a curious feeling" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 2, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0006", "chunk_id": "C0006", "start_para": 72, "end_para": 91, "sampling_reason": "middle_stratified", "char_len": 2001, "text_preview": "“Come, there’s no use in crying like that!” said Alice to herself, rather sharply; “I advise you to leave off this minute!” She generally gave herself very good advice, (though she very seldom followed it), and sometimes...", "chunk_index": 6, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 1.65, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "But it’s no use now", "EAT ME", "Well, I’ll eat it", "Which way? Which way", "Curiouser and curiouser" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 5, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0007", "chunk_id": "C0007", "start_para": 91, "end_para": 105, "sampling_reason": "middle_stratified", "char_len": 2093, "text_preview": "for the moment she quite forgot how to speak good English); “now I’m opening out like the largest telescope that ever was! Good-bye, feet!” (for when she looked down at her feet, they seemed to be almost out of sight, th...", "chunk_index": 7, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 0.65, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "a great girl like you" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0008", "chunk_id": "C0008", "start_para": 105, "end_para": 118, "sampling_reason": "middle_stratified", "char_len": 2177, "text_preview": "in one hand and a large fan in the other: he came trotting along in a great hurry, muttering to himself as he came, “Oh! the Duchess, the Duchess! Oh! won’t she be savage if I’ve kept her waiting!” Alice felt so desperat...", "chunk_index": 8, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 0.9, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "If you please, sir—", "I’m sure I’m not Ada" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 2, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0009", "chunk_id": "C0009", "start_para": 118, "end_para": 132, "sampling_reason": "middle_stratified", "char_len": 2198, "text_preview": "Improve his shining tail, And pour the waters of the Nile On every golden scale! “How cheerfully he seems to grin, How neatly spread his claws, And welcome little fishes in With gently smiling jaws!” “I’m sure those are...", "chunk_index": 9, "all_reasons": [ "middle_stratified", "high_entity_density" ], "density_score": 0.65, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "and now for the garden" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } }, { "sample_id": "OS0010", "chunk_id": "C0010", "start_para": 132, "end_para": 145, "sampling_reason": "tail_stratified", "char_len": 2155, "text_preview": "As she said these words her foot slipped, and in another moment, splash! she was up to her chin in salt water. Her first idea was that she had somehow fallen into the sea, “and in that case I can go back by railway,” she...", "chunk_index": 10, "all_reasons": [ "tail_stratified", "high_entity_density" ], "density_score": 0.65, "signal_terms": { "chapter_headings": [], "quoted_phrases": [ "Où est ma chatte" ], "named_places": [], "titled_people": [], "organizations": [], "artifacts": [], "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } } ], "stats": { "chunk_count": 10, "sample_count": 10, "counts_by_primary_reason": { "front_loaded": 2, "middle_stratified": 7, "tail_stratified": 1 }, "counts_by_any_reason": { "front_loaded": 2, "high_entity_density": 10, "middle_stratified": 8, "tail_stratified": 1 }, "top_density_chunks": [ { "chunk_id": "C0006", "chunk_index": 6, "density_score": 1.65, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 5, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0004", "chunk_index": 4, "density_score": 1.15, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 3, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0002", "chunk_index": 2, "density_score": 0.9, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 2, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0005", "chunk_index": 5, "density_score": 0.9, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 2, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0008", "chunk_index": 8, "density_score": 0.9, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 2, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0003", "chunk_index": 3, "density_score": 0.65, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0007", "chunk_index": 7, "density_score": 0.65, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0009", "chunk_index": 9, "density_score": 0.65, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0010", "chunk_index": 10, "density_score": 0.65, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 1, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } }, { "chunk_id": "C0001", "chunk_index": 1, "density_score": 0.4, "term_highlights": [], "signal_counts": { "chapter_headings": 0, "quoted_phrases": 0, "named_places": 0, "titled_people": 0, "organizations": 0, "artifacts": 0, "conservative_terms": 0 } } ] }, "resolved_strategy": { "front_count": 2, "tail_count": 1, "middle_radius": 1, "preview_chars": 220 } }