태그

Cloudera CDH 6.3.x 설치 [2부]

앞선 과정에서 환경 세팅이 끝나고 본격적으로 Cloudera Manager를 노드에 설치한다.
만약 [1부]에서 했던 과정을 제대로 하지 않으면 에러가 날 수 있으니 주의하세요

참고 : Cloudera  CDH 6.3.x 설치 [1부]
    ● https://radix7318.blogspot.com/2019/10/cloudera-63x-1.html


▶ Cloudera Manager 설치

1. CDH 6.3 Repository 저장소 추가하기
# yum install yum-utils -y
# yum-config-manager --enable cloudera-manager

2. 오라클 자바 설치 또는 OpenJDK 설치
# rpm -ivh jdk-8u221-linux-x64.rpm
또는 
# sudo yum install oracle-j2sdk1.8
또는
# su -c yum install java-1.8.0-openjdk-devel

3. Cloudera Manager 패키지 설치

- 매니저 노드의 경우
# sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
- 나머지 노드의 경우
# sudo yum install cloudera-manager-daemons cloudera-manager-agent
4. 데이터베이스 설치 (매니저 장비만)

- 클라우데라에서 지원하는 데이터베이스를 사용 할 때
# yum install cloudera-scm-server-db
- MariaDB 인 경우
1. MariaDB 설치
# sudo yum install mariadb-server

2. MariaDB 환경설정
# /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the

key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MariaDB, if you enable the binary log and do not set
#a server_id, MariaDB will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

3. MariaDB 서비스 등록
# sudo systemctl enable mariadb

4. MariaDB 서비스 시작
# sudo systemctl start mariadb

5. MariaDB 초기화
# sudo /usr/bin/mysql_secure_installation

6. mysql-connector-java-5.1.46 다운

7. mysql-connector-java-5.1.46 tar 해제
# tar zxvf mysql-connector-java-5.1.46.tar.gz

8. mysql-connector-java-5.1.46 등록
# sudo mkdir -p /usr/share/java/
# cd mysql-connector-java-5.1.46
# sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar


9. 계정 및 Database 생성
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'root99';
flush privileges;
GRANT ALL privileges ON scm.* TO 'scm'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON amon.* TO 'amon'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON rman.* TO 'rman'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON hue.* TO 'hue'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON metastore.* TO 'hive'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON sentry.* TO 'sentry'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON nav.* TO 'nav'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON navms.* TO 'navms'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;
GRANT ALL privileges ON oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'root99';
FLUSH PRIVILEGES;

10. Cloudera Manager Database 셋팅
# sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h localhost --scm-host root scm root
5. 서비스 기동
# sudo systemctl start cloudera-scm-server (매니저 장비만)
# sudo systemctl start cloudera-scm-agent (전부)
- 기본 DB 사용하는 경우
# sudo systemctl start cloudera-scm-server-db (매니저 장비만)
# sudo systemctl start cloudera-scm-server (매니저 장비만)
# sudo systemctl start cloudera-scm-agent (전부)

6. Cloudera Manager가 올라왔는지 확인
# netstat -nap | grep 7180
tcp        0      0 0.0.0.0:7180            0.0.0.0:*               LISTEN      14413/java  
....
라벨:

댓글

댓글 쓰기