{"id":6139,"date":"2020-07-24T15:00:00","date_gmt":"2020-07-24T15:00:00","guid":{"rendered":"https:\/\/bentego.com\/logstash-ile-elk-uygulamalari-tasarlamak\/"},"modified":"2025-10-20T16:09:43","modified_gmt":"2025-10-20T16:09:43","slug":"logstash-ile-elk-uygulamalari-tasarlamak","status":"publish","type":"post","link":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/","title":{"rendered":"Logstash ile ELK Uygulamalar\u0131 Tasarlamak"},"content":{"rendered":"\n<p>Selamlar herkese bu yaz\u0131mda ELK stack yap\u0131s\u0131nda bulunan Logstash\u2019e giri\u015f yap\u0131p, teori ve uygulamalar ile u\u00e7tan uca Logstash pipelinelar\u0131n\u0131 g\u00f6sterece\u011fim.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Logstash<\/h3>\n\n\n\n<p>Logstash, \u00e7ok say\u0131da kaynaktan veri alabilen, d\u00f6n\u00fc\u015ft\u00fcren ve daha sonra ba\u015fka kaynaklara g\u00f6nderebilen a\u00e7\u0131k kaynakl\u0131 bir veri i\u015fleme hatt\u0131d\u0131r.<\/p>\n<p style=\"text-align: center;\"><strong><img fetchpriority=\"high\" decoding=\"async\" class=\"size-full wp-image-4647 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/logstash.png\" alt=\"\" width=\"782\" height=\"428\" \/>\u015eekil 1:\u00a0<\/strong>Logstash[1]<\/p>\n\n\n\n\n\n<h4 class=\"wp-block-heading\">1.1 Logstash Nas\u0131l \u00c7al\u0131\u015f\u0131r?<\/h4>\n\n\n\n<p>Logstash pipeline\u2019\u0131 temel olarak 3 a\u015famadan olu\u015fur. \u0130nput, Filter ve Output.<br \/>\u0130nputlar : Eventlerin gelmesi, olu\u015fturulmas\u0131d\u0131r.<br \/>Filterlar : Gelen eventlerin \u00fczerinde aksiyonlar, transformasyonlar ger\u00e7ekle\u015ftirirler.<br \/>Outputlar : Eventleri ba\u015fka bir yere aktar\u0131rlar.<\/p>\n<p style=\"text-align: center;\"><strong><img decoding=\"async\" class=\"size-full wp-image-4648 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/logstash_internal_architecture.png\" alt=\"\" width=\"600\" height=\"180\" \/><\/strong><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 2<\/strong>\u00a0: Logstash pipeline[2]<\/p>\n\n\n\n\n\n<p>\u00d6rnek bir Logstash config dosyas\u0131n\u0131n yap\u0131s\u0131 a\u015fa\u011f\u0131da g\u00f6sterilmi\u015ftir. \u0130lerleyen k\u0131s\u0131mlarda daha detayl\u0131 olarak de\u011finilecektir.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>input {\n  tcp {\n    port =&gt; 5000\n    type =&gt; syslog\n  }\n  udp {\n    port =&gt; 5000\n    type =&gt; syslog\n  }\n}\n\nfilter {\n  if [type] == \"syslog\" {\n    grok {\n      match =&gt; { \"message\" =&gt; \"%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{GREEDYDATA:syslog_message}\" }\n      add_field =&gt; [ \"received_at\", \"%{@timestamp}\" ]\n      add_field =&gt; [ \"received_from\", \"%{host}\" ]\n    }\n    date {\n      match =&gt; [ \"syslog_timestamp\", \"MMM  d HH:mm:ss\", \"MMM dd HH:mm:ss\" ]\n    }\n  }\n}\n\noutput {\n  elasticsearch { hosts =&gt; [\"localhost:9200\"] }\n  stdout { codec =&gt; rubydebug }\n}<\/code><\/pre>\n\n\n\n<p>Logstash\u2019in iki t\u00fcr config dosyas\u0131 vard\u0131r: Birincisi Standart conf dosyas\u0131, pipelinenin kendisini yani input, filter ve output\u2019un tan\u0131mland\u0131\u011f\u0131 k\u0131s\u0131m. \u0130kinicisi ise Logstash pipelinenin ba\u015flat\u0131lmas\u0131n\u0131 ve \u00e7al\u0131\u015fma \u015feklini etkileyen se\u00e7enekleri belirten yml conf dosyalar\u0131. Bu yaz\u0131da Logstash\u2019in b\u00fcy\u00fck bir k\u0131sm\u0131n\u0131 olu\u015fturan standart config dosyas\u0131 \u00fczerinden uygulamalar g\u00f6sterilecektir.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">1.2 Pluginler<\/h4>\n\n\n\n<p><strong>Input Plugin :\u00a0<\/strong>Datan\u0131n Logstash\u2019e al\u0131nd\u0131\u011f\u0131 k\u0131s\u0131m. En \u00e7ok kullan\u0131lan Logstash input pluginleri \u015funlard\u0131r.<\/p>\n\n\n\n<p><strong>File<\/strong>\u00a0: Filesystem \u00fczerindeki dosya\/lardan loglar\u0131 okur.<br \/><strong>Redis<\/strong>\u00a0: Redis objesinden loglar\u0131 okur.<br \/><strong>Stdin<\/strong>: Standart inputtan (command line) okur.<br \/><strong>S3<\/strong>: Amazon S3\u2019den loglar\u0131 okur.<br \/>Rabbitmq, Mongodb, Kafka gibi input pluginleri de mevcuttur.<\/p>\n\n\n\n<p><strong>Filter Plugin :\u00a0<\/strong>Input k\u0131sm\u0131ndan gelen loglar\u0131n \u00fczerinde i\u015flemlerin(D\u00f6n\u00fc\u015ft\u00fcrme, parse etme, silme vb) yap\u0131ld\u0131\u011f\u0131 k\u0131s\u0131m. En \u00e7ok kullan\u0131lan filter pluginleri \u015funlard\u0131r.<\/p>\n\n\n\n<p><strong>Grok<\/strong>: Gelen metni ayr\u0131\u015ft\u0131r\u0131r ve yap\u0131land\u0131r\u0131r. Grok, Logstash\u2019de yap\u0131land\u0131r\u0131lmam\u0131\u015f loglar\u0131 yap\u0131land\u0131r\u0131lm\u0131\u015f ve sorgulanabilir bir \u015feye ayr\u0131\u015ft\u0131rman\u0131n en iyi yoludur. Logstash\u2019de built in olarak bulunan yakla\u015f\u0131k 120 pattern vard\u0131r. \u0130p arama, Say\u0131 arama Hostname arama vs.<br \/>A\u015fa\u011f\u0131da \u00f6rnek 4 pattern g\u00f6sterilmi\u015ftir.\u00a0<br \/>UNIXPATH (\/([\\w_%!$@:.,+~-]+|\\.)*)+<br \/>CISCOMAC (?:(?:[A-Fa-f0-9]{4}.){2}[A-Fa-f0-9]{4})<br \/>WINDOWSMAC (?:(?:[A-Fa-f0-9]{2}-){5}[A-Fa-f0-9]{2})<br \/>COMMONMAC (?:(?:[A-Fa-f0-9]{2}:){5}[A-Fa-f0-9]{2})<\/p>\n\n\n\n<p><strong>Mutate:<\/strong>\u00a0Fieldlerde genel d\u00f6n\u00fc\u015f\u00fcmler yapabilir. Eventlerdeki fieldleri yeniden adland\u0131rabilir, kald\u0131rabilir, tiplerini de\u011fi\u015ftirebilirsiniz.<\/p>\n\n\n\n<p><strong>Outputs<\/strong>\u00a0: Logstash pipelinenin son a\u015famas\u0131d\u0131r. Bir event birden fazla outputtan ge\u00e7ebilir. Yayg\u0131n olarak kullan\u0131lan baz\u0131 outputlar \u015funlard\u0131r:<br \/><strong>Elasticsearch<\/strong>\u00a0: loglar\/eventler Elasticsearch\u2019e g\u00f6nderir. Verilerinizi sorgulanabilir bir bi\u00e7imde kaydetmeyi sa\u011flar.<br \/><strong>File<\/strong>\u00a0: Dosya sisteminizdeki bir dosyaya yazar.<br \/><strong>Stdout<\/strong>\u00a0: Standart output\u2019a yazar.<br \/><strong>Kafka<\/strong>\u00a0: Bir Kafka topic \u00fczerine yazar.<\/p>\n\n\n\n<p><strong>Di\u011fer pluginler<\/strong><br \/><strong>Azure event hubs<\/strong>\u00a0: Azure\u2019den eventler al\u0131r.<br \/><strong>Cloudwatch<\/strong>\u00a0: AWS Cloudwatch Api\u2019dan eventleri al\u0131r.<br \/><strong>Github<\/strong>\u00a0: Github\u2019dan eventler al\u0131r.<br \/><strong>Graphite<\/strong>: Graphiteden metricleri al\u0131r.<br \/><strong>http<\/strong>\u00a0: http ve https \u00fczerinden eventler al\u0131r.<br \/><strong>Jdbc<\/strong>\u00a0: Jdbc \u00fczerinden eventleri al\u0131r.<br \/><strong>Rabbitmq<\/strong>\u00a0: Rabbitmq \u00fczerinden eventleri al\u0131r.<br \/><strong>Tcp-udp<\/strong>\u00a0: tcp ve udp \u00fczerinden eventler okunur veya yaz\u0131l\u0131r.<br \/><strong>Twitter<\/strong>\u00a0: Twitter streaming api \u00fczerinden eventler al\u0131n\u0131r.<br \/><strong>Email<\/strong>\u00a0: eventler email server \u00fczerinden g\u00f6nderilir.<br \/><strong>\u0130nfluxdb<\/strong>\u00a0: eventler influxdb\u2019ye yaz\u0131l\u0131r.<br \/><strong>Mongodb<\/strong>\u00a0: eventler mongodb\u2019ye yaz\u0131l\u0131r.<br \/><strong>Solr_http<\/strong>\u00a0: loglar solr\u2019da indexlenir ve saklan\u0131r.<br \/><strong>Webhdfs<\/strong>\u00a0: loglar hdfs\u2019e yaz\u0131l\u0131r.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">1.3 Logstash Terimleri<\/h4>\n\n\n\n<p><strong>Codec :\u00a0<\/strong>Codec eklentisi bir verinin g\u00f6r\u00fcn\u00fcm\u00fcn\u00fc de\u011fi\u015ftirir. Codec bile\u015fenleri esasen bir inputun veya outputun par\u00e7as\u0131 olarak \u00e7al\u0131\u015fabilen stream filtreleridir.<\/p>\n\n\n\n<p><strong>Delimiter :<\/strong>\u00a0Dosyadan okuma yap\u0131l\u0131rken delimiterin ne olaca\u011f\u0131n\u0131 belirtir. Default delimiter new line ( \\n ) karakteridir.<\/p>\n\n\n\n<p><strong>Path :\u00a0<\/strong>Input veya Output Dosya yolunu belirtti\u011fimiz k\u0131s\u0131m.<\/p>\n\n\n\n<p><strong>Start_position :<\/strong>\u00a0Okuma esnas\u0131nda okuma i\u015flemine nereden ba\u015flanaca\u011f\u0131n\u0131 bildiren keyword. Beginning ve end olmak \u00fczere iki adet de\u011fer al\u0131r.<\/p>\n\n\n\n<p><strong>Tags :\u00a0<\/strong>eventlere tag ekleyebilirsiniz. Bu tagleri daha sonra arama veya d\u00f6n\u00fc\u015f\u00fcm yapmak i\u00e7in kullanabilirsiniz<\/p>\n\n\n\n<p>\u00d6rnek tags =&gt;[\u201cdilisim\u201d]<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>filter{\nif \"dilisim\" in tags[]{\n\tdrop();\n} }\n<\/code><\/pre>\n\n\n\n<p><strong>Metadata :\u00a0<\/strong>Eventlere metadata ekleyebilirsiniz. Bunlar eventlerin i\u00e7inde do\u011frudan bulunmaz ancak y\u00f6nlendirme yaparken kullan\u0131labilir. \u00d6rnek: 3 inputunuz var (Kafka, file, tcp) ve 4 outputunuz var. Hangisi nereye gidecek bunu belirtmek i\u00e7in metadatalardan yard\u0131m al\u0131nabilir.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">1.4 Logstash \u015eartl\u0131 \u0130fadeler<\/h4>\n\n\n\n<p>Bazen bir eventi\u00a0belirli \u015fartlar alt\u0131nda filtrelemek veya outputa aktarmak istersiniz. Bunun i\u00e7in bir conditional ifadeleri kullanabilirsiniz. Logstash\u2019in \u015fartl\u0131 ifadeleri programlama dillerinde oldu\u011fu gibi \u00e7al\u0131\u015f\u0131r. Temel syntax\u0131 \u015fu \u015fekildedir.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" class=\"wp-image-542\" src=\"http:\/\/blog.dilisim.com\/wp-content\/uploads\/2020\/05\/image-43.png\" alt=\"\" \/>\n<figcaption><strong>\u015eekil 3<\/strong>\u00a0: Logstash \u015eartl\u0131 \u0130fadeler[3]<\/figcaption>\n<\/figure>\n\n\n\n<p>Operat\u00f6rler olarak programlama dillerinden de bildi\u011fimiz ==, != ,&lt; ,&gt; , &lt;= , &gt;=, in, not in , and , or , nand , xor , ve ! kullan\u0131l\u0131yor.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">1.5 Logstash Kurulumu<\/h4>\n\n\n\n<p>\u0130lk olarak https:\/\/www.elastic.co\/downloads\/logstash adresine girip istedi\u011fimiz Logstash versiyonunu ve i\u015fletim sistemimiz se\u00e7ip indiriyoruz. Bu yaz\u0131da Logstash\u2019in 6.8.0 s\u00fcr\u00fcm\u00fc kullan\u0131lm\u0131\u015ft\u0131r. Dosyalar\u0131 indirdikten sonra ilk olarak cmd \u00fczerinden Logstash\u2019\u0131n oldu\u011fu klas\u00f6re gidip command line \u00fczerinden \u201cbin\/logstash\u201d komutu ile Logstash\u2019i \u00e7al\u0131\u015ft\u0131r\u0131yoruz.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">1.6 Logstash ile uygulamalar<\/h4>\n\n\n\n<p><strong>1.6.1 Standart input \u2013 Standart output<\/strong><\/p>\n\n\n\n<p>Logstash ile yapaca\u011f\u0131m\u0131z ilk uygulamada command line \u00fczerinden girdi\u011fimiz eventi yine command line ile okuyaca\u011f\u0131z.<\/p>\n\n\n\n<p>\u0130lk olarak cmd ile Logstash\u2019in oldu\u011fu directorye gidiyoruz ve istedi\u011fimiz yere logstash.conf isimli dosyay\u0131 olu\u015fturuyoruz. Daha sonra conf dosyam\u0131z\u0131 a\u00e7\u0131p a\u015fa\u011f\u0131daki sat\u0131rlar\u0131 ekliyoruz.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>input {\n\tstdin {\n\n\t}\n}\n#filter k\u0131sm\u0131n\u0131 eklememize \u015fu an i\u00e7in gerek yok. Filter k\u0131sm\u0131 olmasa bile logstash \u00e7al\u0131\u015fabilmektedir. Ald\u0131\u011f\u0131 veriyi oldu\u011fu gibi aktar\u0131r.\n\noutput {\n\tstdout {\n\n\t}\n}\n<\/code><\/pre>\n\n\n\n<p>Daha sonra dosyam\u0131z\u0131 kaydedip \u00e7\u0131k\u0131yoruz ve\u00a0<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>bin\/logstash -f config\/pipelines\/pipeline.conf<\/p>\n<\/blockquote>\n\n\n\n<p>komutu ile ilk Logstash uygulamam\u0131z\u0131 \u00e7al\u0131\u015ft\u0131r\u0131yoruz. Logstash\u2019in ba\u015flamas\u0131 45-50 saniye kadar s\u00fcrebilmektedir. \u201cSuccessfully started\u201d yaz\u0131s\u0131n\u0131 g\u00f6rd\u00fckten sonra \u201cSelamlar dilisim\u201d yaz\u0131p g\u00f6nderiyoruz. Logstash ise cevap olarak bize g\u00f6nderdi\u011fimiz mesaj\u0131n yan\u0131 s\u0131ra 3 adet bilgi daha d\u00f6n\u00fcyor. Bunlar Logstash\u2019in \u00e7al\u0131\u015ft\u0131\u011f\u0131 host ad\u0131, bu i\u015flemin yap\u0131ld\u0131\u011f\u0131 tarih ve versiyon numaras\u0131d\u0131r.<\/p>\n<p style=\"text-align: center;\"><strong><img decoding=\"async\" class=\"size-full wp-image-4649 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0.png\" alt=\"\" width=\"940\" height=\"284\" \/>\u015eekil 4<\/strong>\u00a0: \u0130lk uygulamam\u0131z\u0131n console g\u00f6r\u00fcnt\u00fcs\u00fc<\/p>\n\n\n\n\n\n<p>Peki plain texti d\u00fczg\u00fcn bir \u015fekilde iletti ya inputumuz json format\u0131nda olsayd\u0131 o zaman ne olacakt\u0131 ? Name fieldinin de\u011feri Tolga olacak \u015fekilde bir input g\u00f6nderelim.<\/p>\n<p style=\"text-align: center;\"><strong><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4650 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-1.png\" alt=\"\" width=\"653\" height=\"141\" \/><\/strong><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 5<\/strong>\u00a0: Logstash\u2019in plain text ayarlar\u0131yla Json g\u00f6sterimi<\/p>\n\n\n\n\n\n<p>Biz name\u2019i bir field olarak istemi\u015ftik ancak bu direkt olarak mesaj\u0131n i\u00e7ine katt\u0131. Peki ne yapabiliriz ? Yukar\u0131da bahsetti\u011fim codec pluginini kullanabiliriz. \u015eimdi hem json format\u0131nda input alan hem de ald\u0131\u011f\u0131 inputu hem command line \u00fczerinden bize d\u00f6nd\u00fcren hem de bir text dosyas\u0131na kaydeden bir uygulama yapal\u0131m. Config dosyam\u0131z\u0131n i\u00e7ine girip a\u015fa\u011f\u0131daki \u015fekilde de\u011fi\u015fiklik yapal\u0131m.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>input {\n\tstdin {\n\t\tcodec =&gt; json\n\t}\n}\n\noutput {\n\tstdout {\n\n        }\n\n\tfile {\n\t\tpath =&gt; \"tempoutput.txt\"\n\t}\n}\n<\/code><\/pre>\n\n\n\n<p>Codec kullanarak art\u0131k json tipindeki loglar\u0131m\u0131z\u0131\/eventlerimizi Logstash\u2019e tan\u0131tabilecek ve file plugin kullanarak bunlar\u0131 bir dosyaya kaydedebilece\u011fiz. \u015eimdi tekrar Logstash pipelinem\u0131z\u0131 aya\u011fa kald\u0131ral\u0131m ve az \u00f6nceki inputu tekrar girelim.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4651 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-2.png\" alt=\"\" width=\"885\" height=\"187\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 6<\/strong>\u00a0: Logstash Json g\u00f6sterimi<\/p>\n\n\n\n\n\n<p>G\u00f6rd\u00fc\u011f\u00fcn\u00fcz gibi bu sefer name ad\u0131nda bir field ve Tolga ad\u0131nda bu fielde ait bir value elde ettik. Son sat\u0131rdaki infoya dikkat edelim. Burada bize belirtilen pathte bir dosya a\u00e7\u0131ld\u0131\u011f\u0131n\u0131 ve dosyaya yazma i\u015fleminin yap\u0131ld\u0131\u011f\u0131ndan bahsediyor.\u00a0<\/p>\n\n\n\n<p>\u015eimdi bu sefer i\u00e7in i\u00e7ine ba\u015fka uygulamalar katal\u0131m. Postman \u00fczerinden json tipinde eventler yollay\u0131p bunlar\u0131 yine cmdden izleyelim.<\/p>\n\n\n\n<p>Bunun i\u00e7in yukar\u0131daki config dosyam\u0131z\u0131n input k\u0131sm\u0131na http pluginini ekliyoruz ve daha sonra uygulamam\u0131z\u0131 tekrar ba\u015flat\u0131p Postman clientimizi a\u00e7\u0131yoruz.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>http {\n\t host =&gt; \"127.0.0.1\"\n\t port =&gt; 8080\n }\n#host ve port bilgimizi yukar\u0131daki \u015fekilde ayarl\u0131yoruz. Port istedi\u011finiz bir de\u011fer olabilir ( ba\u015fka bir uygulama taraf\u0131ndan kullan\u0131lmamas\u0131 gerekir ).<\/code><\/pre>\n\n\n\n<p>Postman clientimizi a\u00e7\u0131ktan sonra host ve port bilgilerini girip sa\u011f taraftan jsonu se\u00e7iyoruz ve g\u00f6ndermek istedi\u011fimiz eventleri yaz\u0131yoruz. Name ve Peoplecount fieldlerinden olu\u015fan bir json g\u00f6nderdim.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4652 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-3.png\" alt=\"\" width=\"976\" height=\"259\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 7<\/strong>\u00a0: Postman\u0131n i\u00e7 g\u00f6r\u00fcnt\u00fcs\u00fc<\/p>\n\n\n\n\n\n<p>Daha sonra console \u00fczerinden kontrol etti\u011fimzde bu sefer yukar\u0131daki outputlardan \u00e7ok daha fazla say\u0131da field oldu\u011funu g\u00f6zlemleyebiliriz. Bunun sebebi ise \u00fc\u00e7\u00fcnc\u00fc parti bir uygulama\/agent kullanm\u0131\u015f olmam\u0131zdan dolay\u0131d\u0131r.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4653 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-4.png\" alt=\"\" width=\"668\" height=\"415\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 8<\/strong>\u00a0: Postman agent bilgileri<\/p>\n\n\n\n\n\n<p>\u0130lk iki sat\u0131rda g\u00f6nderdi\u011fimiz de\u011ferleri g\u00f6rebiliyoruz. Geri kalan header k\u0131sm\u0131 ise bizim hangi agenti kulland\u0131\u011f\u0131m\u0131z ve agent \u00fczerinde hangi ayarlar\u0131 yapt\u0131\u011f\u0131m\u0131z hakk\u0131nda bize bilgi veriyor. Bu asl\u0131nda \u00e7ok yararl\u0131 bir bilgidir. Siz kendi serverinize Postman ya da X uygulamas\u0131 \u00fczerinden request at\u0131lmas\u0131n\u0131 istemiyorsan\u0131z bunu filter k\u0131sm\u0131nda if sorgusu ile kontrol ederek istemedi\u011finiz yerlerden gelen requestleri engelleyebilirsiniz.<\/p>\n\n\n\n<p><strong>1.6.2 Logstash \u2013 TCP uygulamas\u0131<\/strong><\/p>\n\n\n\n<p>Bu uygulamada bir port \u00fczerinden gelen veriyi localde bir dosyaya kaydedece\u011fiz. Bu sefer input, filter ve outputu par\u00e7alayarak g\u00f6sterece\u011fim.\u00a0<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>input {\n\t\ttcp {\n\t\t\tport =&gt; 5400\n\t\t\tcodec =&gt; json\n\t\t\t}\n}\n<\/code><\/pre>\n\n\n\n<p>5400 portuna gelen loglar\u0131 tcp \u00fczerinden ile Logstash\u2019e aktaraca\u011f\u0131z. Bunun i\u00e7in tcp pluginini kullan\u0131yoruz. Json tipinde veriler aktaraca\u011f\u0131m\u0131z i\u00e7in codec belirttik ve son olarak port olarak 5400 se\u00e7tim. Siz ba\u015fka bir port ile deneyebilirsiniz.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>filter {\n\tmutate {\n\t\tconvert =&gt; {\n\t\t\t\"age\" =&gt; \"integer\"\n\t\t\t\"_id\" =&gt; \"integer\"\n\t\t}\n\t\tremove_field =&gt; [\"host\",\"@version\",\"@timestamp\",\"port\"]\n\t\t}\n}<\/code><\/pre>\n\n\n\n<p>Filter k\u0131sm\u0131nda ise bir tak\u0131m i\u015flemler yapt\u0131k. \u0130lk olarak age ve _id fieldleri e\u011fer string tipinde ( \u201cx\u201d ) geliyorsa bunlar\u0131 integer\u2019a \u00e7evirdik. \u0130lk uygulamam\u0131zda g\u00f6rm\u00fc\u015ft\u00fck Logstash baz\u0131 fieldler ekliyordu. Bunlar\u0131 bu sefer d\u0131\u015farda b\u0131rakmak istedim bunun i\u00e7in mutate plugininin remove_field \u00f6zelli\u011finden faydaland\u0131m. Silmek istedi\u011fim alanlar\u0131 i\u00e7ine string olarak ekledim.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>output {\n\tstdout {\n\t\t\t\n\t}\n}\n<\/code><\/pre>\n\n\n\n<p>Daha sonra tekrar commandline \u00fczerinden g\u00f6rmek istedi\u011fim i\u00e7in output k\u0131sm\u0131n\u0131 bu \u015fekilde d\u00fczelttim.<\/p>\n\n\n\n<p>Peki 5400 portuna nas\u0131l veri g\u00f6nderece\u011fiz? Ben bunun i\u00e7in netcat\u2019i kulland\u0131m. Netcat, a\u011f\u0131 okuyan ve TCP veya UDP ileti\u015fim kurallar\u0131n\u0131 kullanarak a\u011f ba\u011flant\u0131lar\u0131n\u0131 yazan bir hizmetdir.\u00a0<\/p>\n\n\n\n<p>\u0130lk olarak bir json dosyas\u0131 olu\u015fturup a\u015fa\u011f\u0131daki sat\u0131rlar\u0131 dosyan\u0131n i\u00e7ine ekleyip kaydettim.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\"_id\":\"1\",\"name\":\"Ali\",\"age\":\"10\",\"city\":\"istanbul\"}\n{\"_id\":\"2\",\"name\":\"Veli\",\"age\":\"22\",\"city\":\"bursa\"}\n{\"_id\":\"3\",\"name\":\"Hasan\",\"age\":\"34\",\"city\":\"bayburt\"}\n{\"_id\":\"4\",\"name\":\"Elif\",\"age\":\"11\",\"city\":\"bal\u0131kesir\"}\n{\"_id\":\"5\",\"name\":\"Gizem\",\"age\":\"53\",\"city\":\"adana\"}\n<\/code><\/pre>\n\n\n\n<p>Daha sonra tekrar bir cmd a\u00e7\u0131p yukar\u0131daki x.json dosyam\u0131z\u0131n oldu\u011fu konuma geldim ve a\u015fa\u011f\u0131daki komut ile dosyam\u0131z\u0131 netcat ile 5400 portuna aktarm\u0131\u015f oldum.<\/p>\n\n\n\n<p>nc localhost 5400 &lt; original.json<\/p>\n\n\n\n<p>Daha sonra Logstash\u2019i \u00e7al\u0131\u015ft\u0131rd\u0131\u011f\u0131m terminal ekran\u0131na d\u00f6nd\u00fc\u011f\u00fcm zaman sonu\u00e7lar\u0131n ekrana 1 saniyeden k\u0131sa bir s\u00fcrede gelmi\u015f oldu\u011funu g\u00f6rd\u00fcm.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4654 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-5.png\" alt=\"\" width=\"988\" height=\"585\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 9<\/strong>\u00a0: Json dosyas\u0131n\u0131n Tcp plugininde g\u00f6sterimi<\/p>\n\n\n\n\n\n<p><strong>1.6.3 Logstash \u2013 Twitter Uygulamas\u0131<\/strong><\/p>\n\n\n\n<p>Bu uygulamam\u0131zda Logstash Twitter input plugini kullaranak Twitter API ile ileti\u015fime ge\u00e7ip belirtti\u011fimiz hashtage ait tweetleri \u00e7ekip bir dosyaya kaydedece\u011fiz.<\/p>\n\n\n\n<p>Config dosyam\u0131z a\u015fa\u011f\u0131daki gibidir. Bunun i\u00e7in ilk olarak twitter developer accountu a\u00e7mal\u0131 ve a\u015fa\u011f\u0131daki 4 adet key\u2019e sahip olmal\u0131s\u0131n\u0131z. Ben kendi keylerimi payla\u015fam\u0131yorum maalesef.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>input {\n\ttwitter {\n\t\tconsumer_key =&gt; \"xxxx\"\n\t\tconsumer_secret =&gt; \"xxx\"\n\t\toauth_token =&gt; \"xxxx\"\n\t\toauth_token_secret =&gt; \"xxxx\"\n\t\tkeywords =&gt; [ \"big data\"]\n\t\tfull_tweet =&gt; true\n\t\tignore_retweets =&gt; true\n}\n}\n\noutput {\nfile {\n\tpath =&gt; \"tweetsnew.json\"\n\n}\n}\n\n\n<\/code><\/pre>\n\n\n\n<p>Burada keywords fieldi aramak istedi\u011fimiz hashtag\u0131, full_tweet fieldi o tweete air t\u00fcm detay bilgileri alan ve ignore_retweets ise retweetleri ignore etmemizi sa\u011flayan fieldler. Retweetleri ignore ettik \u00e7\u00fcnk\u00fc bir hashtage at\u0131lan binlerce retweet tweetler \u00fczerinde bir analiz yapmak istedi\u011fimiz zaman bizi yan\u0131ltabilir.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4655 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-6.png\" alt=\"\" width=\"858\" height=\"88\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 10<\/strong>\u00a0: Logstash Twitter Plugini<\/p>\n\n\n\n\n\n<p>Daha sonra json dosyam\u0131za girdi\u011fimizde tweetlerin gelmi\u015f oldu\u011funu g\u00f6rebilece\u011fiz. \u00c7ok fazla field i\u00e7erdi\u011finden 1 tweete ait \u00e7ekilen bilgileri a\u015fa\u011f\u0131da g\u00f6sterebilirim.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\"quote_count\":0,\n\"in_reply_to_screen_name\":null,\n\"filter_level\":\"low\",\n\"possibly_sensitive\":false,\n\"created_at\":\"Thu May 07 11:24:49 +0000 2020\",\"retweet_count\":0,\"@version\":\"1\",\"@timestamp\":\"2020-05-07T11:24:49.000Z\",\n\"is_quote_status\":false,\n\"truncated\":false,\n\"place\":null,\n\"entities\":{\"urls\":[{\"url\":\"https:\/\/t.co\/T6Mvytsqse\",\"expanded_url\":\"https:\/\/bit.ly\/3fcor94\",\"display_url\":\"bit.ly\/3fcor94\",\"indices\":[81,104]}],\"user_mentions\":[],\"symbols\":[],\"hashtags\":[]},\"coordinates\":null,\"id_str\":\"1258357111345553412\",\"timestamp_ms\":\"1588850689146\",\"in_reply_to_status_id_str\":null,\n\"source\":\"&lt;a href=\\\"http:\/\/twitter.com\/download\/iphone\\\" rel=\\\"nofollow\\\"&gt;Twitter for iPhone&lt;\/a&gt;\",\"in_reply_to_user_id\":null,\n\"text\":\"Top Ways in Which Big Data Has Fortified Geolocation Apps - Data Science Central https:\/\/t.co\/T6Mvytsqse\",\"reply_count\":0,\"favorite_count\":0,\"in_reply_to_user_id_str\":null,\"in_reply_to_status_id\":null,\"geo\":null,\"id\":1258357111345553412,\"contributors\":null,\"retweeted\":false,\"user\":{\"description\":null,\"url\":\"http:\/\/about.me\/nivanecu\",\"protected\":false,\"profile_background_image_url_https\":\"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png\",\"statuses_count\":549,\"followers_count\":31,\"contributors_enabled\":false,\"profile_use_background_image\":false,\"created_at\":\"Sat May 28 00:01:42 +0000 2011\",\"is_translator\":false,\"location\":\"Quito, Ecuador\",\"default_profile\":false,\"translator_type\":\"none\",\"profile_background_image_url\":\"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png\",\"screen_name\":\"nivanecu\",\"profile_banner_url\":\"https:\/\/pbs.twimg.com\/profile_banners\/306491885\/1431542810\",\"profile_link_color\":\"0084B4\",\"notifications\":null,\"profile_text_color\":\"000000\",\"id_str\":\"306491885\",\"following\":null,\"name\":\"Nelson Ivan Herrera\",\"utc_offset\":null,\"listed_count\":2,\"profile_sidebar_border_color\":\"000000\",\"verified\":false,\"favourites_count\":83,\"time_zone\":null,\"profile_image_url_https\":\"https:\/\/pbs.twimg.com\/profile_images\/1252324954667978752\/7vofA9XA_normal.jpg\",\"profile_image_url\":\"http:\/\/pbs.twimg.com\/profile_images\/1252324954667978752\/7vofA9XA_normal.jpg\",\"profile_background_color\":\"000000\",\"id\":306491885,\"default_profile_image\":false,\"follow_request_sent\":null,\"profile_background_tile\":false,\"lang\":null,\"friends_count\":43,\"profile_sidebar_fill_color\":\"000000\",\"geo_enabled\":true},\"lang\":\"en\",\"favorited\":false}\n<\/code><\/pre>\n\n\n\n<p><strong>1.6.4 ELK ( Elasticsearch, Logstash, Kibana ) ve Kafka uygulamas\u0131\u00a0<\/strong><\/p>\n\n\n\n<p>Bu uygulamada local disk \u00fczerinde bulunan bir csv dosyas\u0131n\u0131 al\u0131p \u00f6nce Elasticsearch\u2019e g\u00f6nderip Kibana \u00fczerinden kontrol edece\u011fiz daha sonra ayn\u0131 veriyi bir Kafka topicine yazaca\u011f\u0131z.<\/p>\n\n\n\n<p>Elimizde \u201cislem(str)\u201d,\u201dislemtarihi(str)\u201d,\u201dyapankisi(int)\u201d,\u201dbasari_kodu(str)\u201d,\u201dislemsuresi(int)\u201d,\u201doperator_server_definer(str)\u201d s\u00fctunlar\u0131ndan olu\u015fan 3 adet csv dosyas\u0131 var. Bu csv dosyalar\u0131n\u0131n ad\u0131 daily_date.csv \u015feklinde burada date \u00f6nemli \u00e7\u00fcnk\u00fc Elasticsearch\u2019te indexlerken bu isimleri kullanaca\u011f\u0131z.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>input {\n\tfile {\n\t\tpath =&gt; \"\/Users\/tolgadilisim\/Desktop\/event-data\/daily_*.csv\"\n\t\tstart_position =&gt; \"beginning\"\n        }\n}<\/code><\/pre>\n\n\n\n<p>Config dosyas\u0131n\u0131 yine \u00fc\u00e7e b\u00f6lerek a\u00e7\u0131klayaca\u011f\u0131m. \u0130lk olarak input k\u0131sm\u0131nda file plugini kullanarak path\u2019imizi ve okumaya dosyan\u0131n neresinden ba\u015flayaca\u011f\u0131m\u0131z\u0131 belirtiyoruz. Dosyalar\u0131m\u0131z\u0131n hepsi daily_date \u015feklinde oldu\u011fu i\u00e7in daily_* notasyonu bize t\u00fcm dosyalar\u0131m\u0131z\u0131 g\u00f6rmemizi sa\u011flayacak.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>filter {\n\t\tcsv {\n\t\t\tseparator =&gt; \",\"\n\t\t\tcolumns =&gt; [\"islem\",\"islemtarihi\",\"yapankisi\",\"basari_kodu\",\"islemsuresi\",\"operator_server_definer\"]\n\t\t}\n\t\truby {\n\t            code =&gt; \"\n\t                event.set('index_type',event.get('path').split('\/')[-1].gsub('.csv',''))\n\t            \"\n\t        }\n\n\t\tmutate {\n\t\t\tconvert =&gt; {\n\t\t\t\t\"yapankisi\" =&gt; \"integer\"\n\t\t\t\t\"islemsuresi\" =&gt; \"integer\"\n\t\t\t}\n\t\t}\n\n\t        mutate { add_field =&gt; { \"[@metadata][mesaj]\" =&gt; \"%{message}\" } }\n\t\tmutate { add_field =&gt; { \"[@metadata][index_type]\" =&gt; \"%{index_type}\" } }\n                mutate { remove_field =&gt; [\"index_type\"] }\n\n\t\tmutate {\n\t\t\t remove_field =&gt; [\"host\",\"message\",\"@version\",\"@timestamp\",\"path\"]\n\n\t\t }\n\n\n\n}\n<\/code><\/pre>\n\n\n\n<p>Burada ilk olarak csv pluginini kullanarak csv dosyalar\u0131m\u0131z\u0131n columnlar\u0131n\u0131 ve seperatoru belirtiyoruz. Ruby plugini kullanarak her bir csv dosyas\u0131 i\u00e7in dosya pathini al\u0131yoruz sondan \/ i\u015faretine g\u00f6re b\u00f6l\u00fcp .csv ifadesini \u201c\u201d ile de\u011fi\u015ftiriyoruz yani geriye sadece dosya ad\u0131 kal\u0131yor ve bunu bir de\u011fi\u015fkene aktar\u0131yoruz.<\/p>\n\n\n\n<p>\u00d6rnek : daily_22-04-2020.csv \u2013&gt; daily_22-04-2020 olacak ve biz daha sonra bu de\u011fi\u015fkenimizi Elasticsearch\u2019te index olu\u015ftururken kullanaca\u011f\u0131z.<\/p>\n\n\n\n<p>Burada \u00e7ok fazla mutate kulland\u0131k bunlar\u0131 beraber ve ayr\u0131 ayr\u0131 kullanabiliriz. Burada ayr\u0131 ayr\u0131 kullan\u0131m\u0131n\u0131 g\u00f6stermek i\u00e7in bu kadar uzatt\u0131m. \u0130lk mutate plugininde yapankisi ve islems\u00fcresini integer\u2019a \u00e7evirdik. Bunlar\u0131n tipi zaten intti ama olur da veri i\u00e7inde yanl\u0131\u015fl\u0131kla string gelirse diye \u00f6nlem ald\u0131k. Daha sonra bunlar\u0131 Kafka\u2019da g\u00f6stermek i\u00e7in hepsini mesaj isimli bir metadata de\u011fi\u015fkenine aktard\u0131k. Metadatalar datan\u0131n i\u00e7inde do\u011frudan yer almazlar ama message de\u011fi\u015fkeni do\u011frudan yer al\u0131yordu. Hem onu ortadan kald\u0131rm\u0131\u015f olduk hem de tekrar elimizde messagenin sahip oldu\u011fu valueyi tuttuk. Di\u011fer mutatelerde ise istemedi\u011fimiz, Logstash\u2019in kendi ekledi\u011fi fieldleri sildik.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>output {\n\t\telasticsearch {\n\t\t\thosts =&gt; [\"localhost:9200\"]\n\t\t\tindex =&gt; \"%{[@metadata][index_type]}\"\n\t\t\tdocument_type =&gt; \"_doc\"\n\t\t\thttp_compression =&gt; true\n\t\t\t\t}\n\n\t\tkafka {\n\t\t\ttopic_id =&gt; \"tolgakafka\"\n\t\t\tbootstrap_servers =&gt; \"localhost:9092\"\n\n\t\t\tcodec =&gt; plain {\n\t\t\t\tformat =&gt; \"%{[@metadata][mesaj]}\"\n\t\t\t}\n\t\t}\n}\n\n<\/code><\/pre>\n\n\n\n<p>Geldik son k\u0131sm\u0131m\u0131za, burada ilk ba\u015fta Elasticsearch pluginini kullanarak Elasticsearch\u2019in \u00e7al\u0131\u015ft\u0131\u011f\u0131 adresi belirttik. Daha sonra yukar\u0131da dosya isimlerini kesmi\u015ftik hat\u0131rlars\u0131n\u0131z. O kesti\u011fimiz isimleri burada index ad\u0131 olarak kullan\u0131yoruz. Yani bu ne demek ? Her bir csv dosyas\u0131 kendi ad\u0131n\u0131 ta\u015f\u0131yan bir indexi olu\u015fturacak ve veriler ayr\u0131 ayr\u0131 indexlerde tutulacak.<\/p>\n\n\n\n<p>Document type ise default Elasticsearch dok\u00fcman tipidir. G\u00fcncel versiyonlarda _docdur ancak eski Elasticsearch versiyonlar\u0131 kullan\u0131yorsan\u0131z doc gibi farkl\u0131 olabilir de\u011fi\u015ftirmeniz gerekli bu y\u00fczden ayr\u0131ca belirttim \u015fu an kullanmama gerek yok \u00e7\u00fcnk\u00fc ben g\u00fcncel Elasticsearch versiyonu kullan\u0131yorum.\u00a0<\/p>\n\n\n\n<p>Kafka taraf\u0131nda ise yine Kafka plugini yard\u0131m\u0131yla veriyi g\u00f6nderece\u011fimiz Kafka topicinin ad\u0131, Kafka serverlerimizin bilgileri ve ne g\u00f6nderece\u011fimizi codec k\u0131sm\u0131nda belirtiyoruz. Bu da \u00fcstteki message -&gt; metadata mesaj d\u00f6n\u00fc\u015f\u00fcm\u00fc yapt\u0131\u011f\u0131m\u0131z mesaj yani verinin kendisiydi direkt virg\u00fcllerle ayr\u0131lm\u0131\u015f \u015fekilde veriyi Kafka\u2019ya bas\u0131yoruz.<\/p>\n\n\n\n<p>\u015eimdi yukar\u0131da anlatt\u0131klar\u0131m\u0131z\u0131 ad\u0131m ad\u0131m uygulayal\u0131m;<\/p>\n\n\n\n<p>\u0130lk olarak a\u015fa\u011f\u0131daki komut ile bir Kafka topici yaratal\u0131m.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>bin\/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic tolgakafka<\/code><\/pre>\n\n\n\n<p>Daha sonra uygulamam\u0131z\u0131 ba\u015flatabiliriz s\u0131ras\u0131yla \u015fu uygulamalar\u0131 a\u015fa\u011f\u0131daki komutlarla ba\u015flat\u0131yoruz: Elasticsearch -&gt; Kibana -&gt; Zookeeper -&gt; Kafka -&gt; Logstash -&gt; Kafka topic<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>bin\/elasticsearch<\/li>\n<li>bin\/kibana<\/li>\n<li>bin\/zookeeper-server-start.sh config\/zookeeper.properties<\/li>\n<li>bin\/kafka-server-start.sh config\/server.properties<\/li>\n<li>bin\/logstash -f config\/pipelines\/pipeline.conf<\/li>\n<li>bin\/kafka-console-consumer.sh \u2013bootstrap-server localhost:9092 \u2013topic tolgakafka<\/li>\n<\/ol>\n\n\n\n<p>\u0130lk olarak Kibana aray\u00fcz\u00fcm\u00fcze girip \u201cget indices\u201d ile indexlerimizi sorgulad\u0131\u011f\u0131m\u0131zda daily dosyalar\u0131m\u0131z\u0131n isminde indexlerimizin olu\u015ftu\u011funu g\u00f6r\u00fcyoruz.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4657 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-7.png\" alt=\"\" width=\"612\" height=\"115\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 11<\/strong>\u00a0: Elasticsearch indexlerinin Kibana\u2019da g\u00f6sterimi<\/p>\n\n\n\n\n\n<p>Daha sonra indexlerimizin i\u00e7i dolu mu diye sorgu att\u0131\u011f\u0131m\u0131zda ise csv dosyalar\u0131m\u0131z\u0131n i\u00e7eriklerinin ba\u015far\u0131l\u0131 bir \u015fekilde Elasticsearch \u00fczerinden aranabilir hale geldi\u011fini g\u00f6zlemliyoruz.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4656 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-8.png\" alt=\"\" width=\"418\" height=\"558\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 12<\/strong>\u00a0: Kibana sorgu sonucu<\/p>\n\n\n\n\n\n<p>Son olarak Kafka topicimize bakal\u0131m. Kafka plugini kullanarak csv dosyalar\u0131n\u0131n i\u00e7eri\u011fini Kafka topicimize de g\u00f6ndermi\u015f olduk.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-4658 aligncenter\" src=\"https:\/\/bentego.com\/wp-content\/uploads\/2020\/07\/pasted-image-0-9.png\" alt=\"\" width=\"662\" height=\"289\" \/><\/p>\n<p style=\"text-align: center;\"><strong>\u015eekil 13<\/strong>\u00a0: Csv\u2019nin Kafka Topic\u2019e yaz\u0131lmas\u0131<\/p>\n\n\n\n\n\n<p><strong>Kaynak\u00e7a<\/strong><br \/>[1] : elastic.co\/guide\/en\/logstash\/current\/introduction.html\u00a0<br \/>[2] : tutorialspoint.com\/logstash\/images\/logstash_internal_architecture.jpg<br \/>[3] : elastic.co\/guide\/en\/logstash\/current\/event-dependent-configuration.html<\/p>\n\n\n\n<p>Bu yaz\u0131m buraya kadard\u0131. Bir sonraki yaz\u0131mda g\u00f6r\u00fc\u015fmek \u00fczere ?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Selamlar herkese bu yaz\u0131mda ELK stack yap\u0131s\u0131nda bulunan Logstash\u2019e giri\u015f yap\u0131p, teori ve uygulamalar ile u\u00e7tan uca Logstash pipelinelar\u0131n\u0131 g\u00f6sterece\u011fim. 1. Logstash [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":5922,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[87],"tags":[170,171,176,177,178,179,145],"class_list":["post-6139","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","tag-elastic-tr","tag-elastic-search-tr","tag-elk-tr","tag-logstash-tr","tag-pipeline-tr","tag-search-tr","tag-tr-tr"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Logstash ile ELK Uygulamalar\u0131 Tasarlamak - Bentego<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\" \/>\n<meta property=\"og:locale\" content=\"tr_TR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Logstash ile ELK Uygulamalar\u0131 Tasarlamak - Bentego\" \/>\n<meta property=\"og:description\" content=\"Selamlar herkese bu yaz\u0131mda ELK stack yap\u0131s\u0131nda bulunan Logstash\u2019e giri\u015f yap\u0131p, teori ve uygulamalar ile u\u00e7tan uca Logstash pipelinelar\u0131n\u0131 g\u00f6sterece\u011fim. 1. Logstash [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\" \/>\n<meta property=\"og:site_name\" content=\"Bentego\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-24T15:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-10-20T16:09:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png\" \/>\n\t<meta property=\"og:image:width\" content=\"2400\" \/>\n\t<meta property=\"og:image:height\" content=\"1600\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Bentego\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Yazan:\" \/>\n\t<meta name=\"twitter:data1\" content=\"Bentego\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tahmini okuma s\u00fcresi\" \/>\n\t<meta name=\"twitter:data2\" content=\"13 dakika\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\"},\"author\":{\"name\":\"Bentego\",\"@id\":\"https:\/\/bentego.com\/tr\/#\/schema\/person\/0348418b7b0cbca83fdd7a899d54821e\"},\"headline\":\"Logstash ile ELK Uygulamalar\u0131 Tasarlamak\",\"datePublished\":\"2020-07-24T15:00:00+00:00\",\"dateModified\":\"2025-10-20T16:09:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\"},\"wordCount\":2555,\"publisher\":{\"@id\":\"https:\/\/bentego.com\/tr\/#organization\"},\"image\":{\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png\",\"keywords\":[\"elastic\",\"elastic search\",\"elk\",\"logstash\",\"pipeline\",\"search\",\"TR\"],\"articleSection\":[\"Blog\"],\"inLanguage\":\"tr\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\",\"url\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\",\"name\":\"Logstash ile ELK Uygulamalar\u0131 Tasarlamak - Bentego\",\"isPartOf\":{\"@id\":\"https:\/\/bentego.com\/tr\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png\",\"datePublished\":\"2020-07-24T15:00:00+00:00\",\"dateModified\":\"2025-10-20T16:09:43+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#breadcrumb\"},\"inLanguage\":\"tr\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage\",\"url\":\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png\",\"contentUrl\":\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png\",\"width\":2400,\"height\":1600},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/bentego.com\/tr\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Logstash ile ELK Uygulamalar\u0131 Tasarlamak\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/bentego.com\/tr\/#website\",\"url\":\"https:\/\/bentego.com\/tr\/\",\"name\":\"Bentego\",\"description\":\"Turning data into enterprise value\",\"publisher\":{\"@id\":\"https:\/\/bentego.com\/tr\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/bentego.com\/tr\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"tr\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/bentego.com\/tr\/#organization\",\"name\":\"Bentego\",\"url\":\"https:\/\/bentego.com\/tr\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\/\/bentego.com\/tr\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/05\/logo-bentego.svg\",\"contentUrl\":\"https:\/\/bentego.com\/wp-content\/uploads\/2025\/05\/logo-bentego.svg\",\"width\":433,\"height\":109,\"caption\":\"Bentego\"},\"image\":{\"@id\":\"https:\/\/bentego.com\/tr\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/bentego.com\/tr\/#\/schema\/person\/0348418b7b0cbca83fdd7a899d54821e\",\"name\":\"Bentego\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Logstash ile ELK Uygulamalar\u0131 Tasarlamak - Bentego","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/","og_locale":"tr_TR","og_type":"article","og_title":"Logstash ile ELK Uygulamalar\u0131 Tasarlamak - Bentego","og_description":"Selamlar herkese bu yaz\u0131mda ELK stack yap\u0131s\u0131nda bulunan Logstash\u2019e giri\u015f yap\u0131p, teori ve uygulamalar ile u\u00e7tan uca Logstash pipelinelar\u0131n\u0131 g\u00f6sterece\u011fim. 1. Logstash [&hellip;]","og_url":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/","og_site_name":"Bentego","article_published_time":"2020-07-24T15:00:00+00:00","article_modified_time":"2025-10-20T16:09:43+00:00","og_image":[{"width":2400,"height":1600,"url":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png","type":"image\/png"}],"author":"Bentego","twitter_card":"summary_large_image","twitter_misc":{"Yazan:":"Bentego","Tahmini okuma s\u00fcresi":"13 dakika"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#article","isPartOf":{"@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/"},"author":{"name":"Bentego","@id":"https:\/\/bentego.com\/tr\/#\/schema\/person\/0348418b7b0cbca83fdd7a899d54821e"},"headline":"Logstash ile ELK Uygulamalar\u0131 Tasarlamak","datePublished":"2020-07-24T15:00:00+00:00","dateModified":"2025-10-20T16:09:43+00:00","mainEntityOfPage":{"@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/"},"wordCount":2555,"publisher":{"@id":"https:\/\/bentego.com\/tr\/#organization"},"image":{"@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage"},"thumbnailUrl":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png","keywords":["elastic","elastic search","elk","logstash","pipeline","search","TR"],"articleSection":["Blog"],"inLanguage":"tr"},{"@type":"WebPage","@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/","url":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/","name":"Logstash ile ELK Uygulamalar\u0131 Tasarlamak - Bentego","isPartOf":{"@id":"https:\/\/bentego.com\/tr\/#website"},"primaryImageOfPage":{"@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage"},"image":{"@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage"},"thumbnailUrl":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png","datePublished":"2020-07-24T15:00:00+00:00","dateModified":"2025-10-20T16:09:43+00:00","breadcrumb":{"@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#breadcrumb"},"inLanguage":"tr","potentialAction":[{"@type":"ReadAction","target":["https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/"]}]},{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#primaryimage","url":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png","contentUrl":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/06\/Frame-82__.png","width":2400,"height":1600},{"@type":"BreadcrumbList","@id":"https:\/\/bentego.com\/tr\/logstash-ile-elk-uygulamalari-tasarlamak\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/bentego.com\/tr\/"},{"@type":"ListItem","position":2,"name":"Logstash ile ELK Uygulamalar\u0131 Tasarlamak"}]},{"@type":"WebSite","@id":"https:\/\/bentego.com\/tr\/#website","url":"https:\/\/bentego.com\/tr\/","name":"Bentego","description":"Turning data into enterprise value","publisher":{"@id":"https:\/\/bentego.com\/tr\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/bentego.com\/tr\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"tr"},{"@type":"Organization","@id":"https:\/\/bentego.com\/tr\/#organization","name":"Bentego","url":"https:\/\/bentego.com\/tr\/","logo":{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/bentego.com\/tr\/#\/schema\/logo\/image\/","url":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/05\/logo-bentego.svg","contentUrl":"https:\/\/bentego.com\/wp-content\/uploads\/2025\/05\/logo-bentego.svg","width":433,"height":109,"caption":"Bentego"},"image":{"@id":"https:\/\/bentego.com\/tr\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/bentego.com\/tr\/#\/schema\/person\/0348418b7b0cbca83fdd7a899d54821e","name":"Bentego"}]}},"_links":{"self":[{"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/posts\/6139","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/comments?post=6139"}],"version-history":[{"count":1,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/posts\/6139\/revisions"}],"predecessor-version":[{"id":6263,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/posts\/6139\/revisions\/6263"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/media\/5922"}],"wp:attachment":[{"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/media?parent=6139"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/categories?post=6139"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bentego.com\/tr\/wp-json\/wp\/v2\/tags?post=6139"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}