syaka-syakaブログ: fluent-agent-liteを使ってみた

ログファイルをただfluentdに送ってまずは管理できるようにしたいってことで、
fluent-agent-liteを使ってみました。

まずはダウンロード。

# cd /usr/local/

# git clone https://github.com/tagomoris/fluent-agent-lite.git
Initialized empty Git repository in /usr/local/fluent-agent-lite/.git/
remote: Counting objects: 578, done.
remote: Total 578 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (578/578), 160.89 KiB | 185 KiB/s, done.
Resolving deltas: 100% (251/251), done.

install.shがあるようなので実行してみます。

# cd /usr/local/fluent-agent-lite/bin/

# ls
fluent-agent-lite  install.sh

# ./install.sh 
cp: `bin' と `/usr/local/fluent-agent-lite/bin' は同じファイルです
cp: `lib' と `/usr/local/fluent-agent-lite/lib' は同じファイルです
cp: `Makefile.PL' と `/usr/local/fluent-agent-lite/Makefile.PL' は同じファイルです

何だかメッセージが表示されて、うまくいきませんでした。
install.shの中を見てみます。

# cat bin/install.sh 
#!/bin/bash
 
set -e
 
ARG="$1"
CLEAN=
if [ "x"$ARG = "xclean" ]; then
    CLEAN="y"
fi
 
cd $(dirname $0)"/../"
 
SOURCEDIR=$(pwd)
 
if [ "x"$PREFIX = "x" ]; then
    PREFIX=
fi
INSTALLDIR=$PREFIX/usr/local/fluent-agent-lite
 
if [ "x"$PERL_PATH = "x" ]; then
    PERL_PATH="perl"
fi
 
if [ -d $INSTALLDIR -a "x"$CLEAN = "xy" ]; then
    rm -rf $INSTALLDIR
fi
 
mkdir -p $INSTALLDIR
 
cp -rp bin lib Makefile.PL $INSTALLDIR
 
cd $INSTALLDIR
 
curl -s -L http://xrl.us/cpanm > $INSTALLDIR/bin/cpanm
chmod +x $INSTALLDIR/bin/cpanm
$PERL_PATH $INSTALLDIR/bin/cpanm -n -lextlib inc::Module::Install
$PERL_PATH $INSTALLDIR/bin/cpanm -n -lextlib --reinstall --installdeps .
 
cd $SOURCEDIR
 
mkdir -p $PREFIX/etc/init.d
cp package/fluent-agent-lite.init $PREFIX/etc/init.d/fluent-agent-lite
chmod +x $PREFIX/etc/init.d/fluent-agent-lite
 
if [ ! -f $PREFIX/etc/fluent-agent-lite.conf -o "x"$CLEAN = x"y" ]; then
    cp package/fluent-agent-lite.conf $PREFIX/etc/fluent-agent-lite.conf
fi

どうやら/usr/local/配下にインストールしようとしているので、
そこにダウンロードしてきたものを置いていてはいけないようです。
ということで、/usr/local/配下のものをいったん削除して、
違うディレクトリで再度やってみます。

# rm -rf fluent-agent-lite

# cd /tmp/

# git clone https://github.com/tagomoris/fluent-agent-lite.git
Initialized empty Git repository in /tmp/fluent-agent-lite/.git/
remote: Counting objects: 578, done.
remote: Total 578 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (578/578), 160.89 KiB | 137 KiB/s, done.
Resolving deltas: 100% (251/251), done.

# cd fluent-agent-lite

# ./bin/install.sh 
--> Working on inc::Module::Install
Fetching http://www.cpan.org/authors/id/B/BI/BINGOS/Module-Install-1.10.tar.gz ... OK
==> Found dependencies: YAML::Tiny
--> Working on YAML::Tiny
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/YAML-Tiny-1.63.tar.gz ... OK
==> Found dependencies: Module::Build::Tiny
--> Working on Module::Build::Tiny
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/Module-Build-Tiny-0.037.tar.gz ... OK
==> Found dependencies: TAP::Harness::Env, ExtUtils::Helpers, ExtUtils::InstallPaths, ExtUtils::Config
--> Working on TAP::Harness::Env
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/Test-Harness-3.32.tar.gz ... OK
Configuring Test-Harness-3.32 ... OK
Building Test-Harness-3.32 ... OK
Successfully installed Test-Harness-3.32
--> Working on ExtUtils::Helpers
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Helpers-0.022.tar.gz ... OK
Configuring ExtUtils-Helpers-0.022 ... OK
Building ExtUtils-Helpers-0.022 ... OK
Successfully installed ExtUtils-Helpers-0.022
--> Working on ExtUtils::InstallPaths
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-InstallPaths-0.010.tar.gz ... OK
Configuring ExtUtils-InstallPaths-0.010 ... OK
==> Found dependencies: ExtUtils::Config
--> Working on ExtUtils::Config
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Config-0.008.tar.gz ... OK
Configuring ExtUtils-Config-0.008 ... OK
Building ExtUtils-Config-0.008 ... OK
Successfully installed ExtUtils-Config-0.008
Building ExtUtils-InstallPaths-0.010 ... OK
Successfully installed ExtUtils-InstallPaths-0.010
Configuring Module-Build-Tiny-0.037 ... OK
Building Module-Build-Tiny-0.037 ... OK
Successfully installed Module-Build-Tiny-0.037
Configuring YAML-Tiny-1.63 ... OK
Building YAML-Tiny-1.63 ... OK
Successfully installed YAML-Tiny-1.63
Configuring Module-Install-1.10 ... OK
==> Found dependencies: File::Remove, Module::ScanDeps
--> Working on File::Remove
Fetching http://www.cpan.org/authors/id/A/AD/ADAMK/File-Remove-1.52.tar.gz ... OK
Configuring File-Remove-1.52 ... OK
Building File-Remove-1.52 ... OK
Successfully installed File-Remove-1.52
--> Working on Module::ScanDeps
Fetching http://www.cpan.org/authors/id/R/RS/RSCHUPP/Module-ScanDeps-1.14.tar.gz ... OK
Configuring Module-ScanDeps-1.14 ... OK
Building Module-ScanDeps-1.14 ... OK
Successfully installed Module-ScanDeps-1.14
Building Module-Install-1.10 ... OK
Successfully installed Module-Install-1.10
9 distributions installed
--> Working on .
Configuring /usr/local/fluent-agent-lite ... OK
==> Found dependencies: Log::Minimal, Data::MessagePack
--> Working on Log::Minimal
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/Log-Minimal-0.19.tar.gz ... OK
Configuring Log-Minimal-0.19 ... OK
Building Log-Minimal-0.19 ... OK
Successfully installed Log-Minimal-0.19
--> Working on Data::MessagePack
Fetching http://www.cpan.org/authors/id/G/GF/GFUJI/Data-MessagePack-0.48.tar.gz ... OK
Configuring Data-MessagePack-0.48 ... OK
Building Data-MessagePack-0.48 ... OK
Successfully installed Data-MessagePack-0.48
<== Installed dependencies for .. Finishing.
2 distributions installed

無事にインストールできたようです。
設定ファイルは/etc/fluent-agent-lite.confです。
まずはfluentdのtagを送りたいログに以下のようにひもづけます。

### LOGS: tag /path/to/log/file
# 
LOGS=$(cat <<"EOF"
xxx /tmp/service/service.log
yyy /tmp/service/error.log
EOF
)

次に、プライマリーとセカンダリーのfluentdサーバーを記述すればとりあえず完了です。

# SERVERNAME[:PORTNUM]
# port number is optional (default: 24224)
PRIMARY_SERVER="10.100.21.14"

# secondary server setting is optional...
SECONDARY_SERVER="10.100.21.15"

以下のコマンドで起動します。

# /etc/init.d/fluent-agent-lite start

無事に起動しました。
起動時のログはデフォルトの設定ファイルだと以下に出てきます。

# cat /tmp/fluent-agent.log 
2014-08-06T10:37:08 [INFO] (11956) starting to forward logs. at /usr/local/fluent-agent-lite/bin/fluent-agent-lite line 269
2014-08-06T10:37:08 [INFO] (11956) tail command line: ['/usr/bin/tail','-n','0','-F','/tmp/service/service.log'] at /usr/local/fluent-agent-lite/bin/fluent-agent-lite line 197
2014-08-06T10:37:08 [INFO] (11969) starting to forward logs. at /usr/local/fluent-agent-lite/bin/fluent-agent-lite line 269
2014-08-06T10:37:08 [INFO] (11969) tail command line: ['/usr/bin/tail','-n','0','-F','/tmp/service/error.log'] at /usr/local/fluent-agent-lite/bin/fluent-agent-lite line 197
2014-08-06T10:37:08 [INFO] (11956) connected to server: ['10.100.21.14',24224] at /usr/local/fluent-agent-lite/bin/../lib/Fluent/AgentLite.pm line 310
2014-08-06T10:37:08 [INFO] (11969) connected to server: ['10.100.21.15',24224] at /usr/local/fluent-agent-lite/bin/../lib/Fluent/AgentLite.pm line 310

使う時の注意点としては、ログをtail -Fで見ているけれども、
ログのポジションは保存していないので、
プロセスが落ちたりしてもう一度起動すると同じ内容が送られてしまったりするようです。

参考URL
http://d.hatena.ne.jp/tagomoris/20120314/1331716214
https://github.com/tagomoris/fluent-agent-lite
http://studio3104.hatenablog.com/entry/20120824/1345795228

syaka-syakaブログ

2014年8月18日月曜日

fluent-agent-liteを使ってみた

0 件のコメント:

コメントを投稿