{"id":186,"date":"2026-11-09T11:23:00","date_gmt":"2026-11-09T11:23:00","guid":{"rendered":"https:\/\/www.akifsen.com.tr\/blog\/?p=186"},"modified":"2026-11-09T11:23:00","modified_gmt":"2026-11-09T11:23:00","slug":"large-file-import-streaming-chunks","status":"publish","type":"post","link":"https:\/\/www.akifsen.com.tr\/blog\/2026\/11\/09\/large-file-import-streaming-chunks\/","title":{"rendered":"B\u00fcy\u00fck Dosya Import&#8217;u: Her \u015eeyi Belle\u011fe Almay\u0131n"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Kullan\u0131c\u0131lar \u00fcr\u00fcn listelerini CSV olarak y\u00fckleyebiliyor.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u0130lk implementasyon:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>List&lt;Product&gt; products = csvReader.readAll(file);\nproductRepository.saveAll(products);<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Test dosyas\u0131 500 sat\u0131r. Her \u015fey \u00e7al\u0131\u015f\u0131yor.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bir m\u00fc\u015fteri 2 milyon sat\u0131rl\u0131k bir dosya y\u00fckl\u00fcyor.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Uygulama birka\u00e7 dakika boyunca yan\u0131t vermiyor. Sonra <code>OutOfMemoryError<\/code> ile \u00e7\u00f6k\u00fcyor. Ayn\u0131 pod&#8217;daki di\u011fer kullan\u0131c\u0131lar\u0131n istekleri de etkileniyor.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">B\u00fct\u00fcn dosyay\u0131 belle\u011fe almamak<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">2 milyon sat\u0131r\u0131n her biri i\u00e7in bir nesne olu\u015fturmak, birka\u00e7 GB bellek demek. \u00dcst\u00fcne JPA&#8217;n\u0131n persistence context&#8217;i her entity&#8217;yi takip ediyor.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u0130lk ad\u0131m dosyay\u0131 ak\u0131\u015f halinde, sat\u0131r sat\u0131r okumak:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>try (var reader = Files.newBufferedReader(path);\n     var parser = CSVFormat.DEFAULT.parse(reader)) {\n    for (CSVRecord record : parser) {\n        \/\/ sat\u0131r\u0131 i\u015fle\n    }\n}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Bellekte ayn\u0131 anda sadece birka\u00e7 sat\u0131r bulunur. Dosya boyutu ne olursa olsun bellek kullan\u0131m\u0131 sabit kal\u0131r.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Par\u00e7a par\u00e7a yazmak<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Her sat\u0131r\u0131 ayr\u0131 bir <code>INSERT<\/code> ve ayr\u0131 bir transaction ile yazmak \u00e7ok yava\u015f. B\u00fct\u00fcn dosyay\u0131 tek transaction&#8217;da yazmak ise saatlerce a\u00e7\u0131k kalan bir transaction ve dev bir rollback riski demek.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Orta yol, sat\u0131rlar\u0131 gruplar halinde i\u015flemek. \u00d6rne\u011fin 1000 sat\u0131rl\u0131k par\u00e7alar:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>read 1000 rows \u2192 validate \u2192 batch insert \u2192 commit \u2192 clear<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">JDBC batch insert, 1000 sat\u0131r\u0131 tek bir round-trip&#8217;te g\u00f6nderir. Hibernate kullan\u0131l\u0131yorsa <code>hibernate.jdbc.batch_size<\/code> ayarlanmal\u0131 ve her par\u00e7adan sonra persistence context temizlenmeli. Aksi halde bellek yine dolar.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u00c7ok b\u00fcy\u00fck hacimlerde PostgreSQL&#8217;in <code>COPY<\/code> komutu, batch insert&#8217;ten de belirgin \u015fekilde h\u0131zl\u0131. Veriyi \u00f6nce ge\u00e7ici bir staging tablosuna <code>COPY<\/code> ile y\u00fcklemek ve sonra SQL ile do\u011frulay\u0131p ana tabloya aktarmak, en h\u0131zl\u0131 y\u00f6ntemlerden biri.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Hatal\u0131 sat\u0131rlar<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">2 milyon sat\u0131r\u0131n 312&#8217;si hatal\u0131. Fiyat alan\u0131 bo\u015f, kategori kodu tan\u0131ms\u0131z, tarih format\u0131 yanl\u0131\u015f.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">B\u00fct\u00fcn import&#8217;u ilk hatada durdurmak, kullan\u0131c\u0131y\u0131 dosyay\u0131 tekrar tekrar d\u00fczeltip y\u00fcklemeye zorlar. Her denemede sadece bir sonraki hatay\u0131 \u00f6\u011frenir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Daha iyi bir deneyim, hatal\u0131 sat\u0131rlar\u0131 atlay\u0131p devam etmek ve sonunda bir hata raporu sunmak:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sat\u0131r 1842: price bo\u015f\nsat\u0131r 9921: category 'ELK-99' bulunamad\u0131<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Hata raporu indirilebilir bir dosya olarak sunulursa, kullan\u0131c\u0131 sadece hatal\u0131 sat\u0131rlar\u0131 d\u00fczeltip tekrar y\u00fckleyebilir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Tabii baz\u0131 durumlarda k\u0131smi import kabul edilemez. Bu karar\u0131n kullan\u0131c\u0131ya veya i\u015f kural\u0131na g\u00f6re verilmesi gerekir.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Asenkron i\u015fleme<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">2 milyon sat\u0131rl\u0131k bir import&#8217;un bir HTTP iste\u011fi i\u00e7inde yap\u0131lmamas\u0131 gerekir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Dosya y\u00fcklenir, bir import i\u015fi olu\u015fturulur ve kullan\u0131c\u0131ya hemen cevap d\u00f6n\u00fcl\u00fcr. \u0130\u015f arka planda i\u015flenir. Kullan\u0131c\u0131 ilerlemeyi g\u00f6rebilir:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Import #4412: 1,240,000 \/ 2,000,000 sat\u0131r i\u015flendi<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Bu i\u015f web isteklerini kar\u015f\u0131layan pod&#8217;larda de\u011fil, ayr\u0131 worker&#8217;larda \u00e7al\u0131\u015ft\u0131r\u0131l\u0131rsa, b\u00fcy\u00fck bir import di\u011fer kullan\u0131c\u0131lar\u0131n deneyimini etkilemez.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Kald\u0131\u011f\u0131 yerden devam<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Import 1,2 milyonuncu sat\u0131rda yar\u0131da kesildi. Deploy oldu veya pod yeniden ba\u015flad\u0131.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ba\u015ftan ba\u015flamak hem zaman kayb\u0131 hem de kopya kay\u0131t riski.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u0130\u015flenen son par\u00e7an\u0131n konumunu kal\u0131c\u0131 olarak kaydetmek, i\u015fin kald\u0131\u011f\u0131 yerden devam etmesini sa\u011flar. Ayr\u0131ca her sat\u0131r\u0131n benzersiz bir anahtarla yaz\u0131lmas\u0131, \u00f6rne\u011fin \u00fcr\u00fcn kodu ile upsert yap\u0131lmas\u0131, ayn\u0131 par\u00e7an\u0131n iki kez i\u015flenmesini zarars\u0131z hale getirir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Spring Batch bu desenleri, yani chunk tabanl\u0131 i\u015fleme, hata atlama, yeniden ba\u015flatma ve ilerleme takibini haz\u0131r olarak sunuyor. Basit import&#8217;lar i\u00e7in a\u011f\u0131r gelebilir ama karma\u015f\u0131k ve kritik veri aktar\u0131mlar\u0131nda tekerle\u011fi yeniden icat etmemeyi sa\u011fl\u0131yor.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Dosya import&#8217;u bir \u00f6zelli\u011fin k\u00fc\u00e7\u00fck bir par\u00e7as\u0131 gibi g\u00f6r\u00fcn\u00fcr.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ama test edilen dosya ile m\u00fc\u015fterinin y\u00fckledi\u011fi dosya aras\u0131ndaki boyut fark\u0131, genellikle production&#8217;daki ilk b\u00fcy\u00fck s\u00fcrpriz oluyor.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>2 milyon sat\u0131rl\u0131k bir CSV&#8217;yi listeye okuyup tek transaction&#8217;da kaydetmek uygulamay\u0131 d\u00fc\u015f\u00fcr\u00fcr. Streaming okuma, par\u00e7a par\u00e7a i\u015fleme, hata raporlama ve kald\u0131\u011f\u0131 yerden devam etmek.<\/p>\n","protected":false},"author":1,"featured_media":185,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-186","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mimari"],"_links":{"self":[{"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/posts\/186","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/comments?post=186"}],"version-history":[{"count":1,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/posts\/186\/revisions"}],"predecessor-version":[{"id":236,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/posts\/186\/revisions\/236"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/media\/185"}],"wp:attachment":[{"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/media?parent=186"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/categories?post=186"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.akifsen.com.tr\/blog\/wp-json\/wp\/v2\/tags?post=186"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}