Hbase客户端API
write by donaldhan, 2020-01-30 22:16引言
上一篇文章Hbase1.2.1 HA环境搭建我,介绍Hbase高可用环境的搭建，今天我们来看一下Hbase客户端的使用。
目录
单机环境搭建
使用的Hbase版本为1.2.1
- 首先配置hostname
donaldhan@ubuntu:/bdp/hbase/data/data/default$ cat /etc/hostname
singleNode
- 配置hosts
donaldhan@ubuntu:/bdp/hbase/data/data/default$ cat /etc/hosts
127.0.0.1 localhost
192.168.5.138 singleNode
# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
donaldhan@ubuntu:/bdp/hbase/data/data/default$
- 配置环境变量
donaldhan@singleNode:~$ cat ~/.bashrc
export JAVA_HOME=/usr/share/jdk1.8.0_191
export JRE_HOME=${JAVA_HOME}/jre
export CLASS_PATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export HADOOP_HOME=/bdp/hadoop/hadoop-2.7.1
export ZOOKEEPER_HOME=/bdp/zookeeper/zookeeper-3.4.12
export HBASE_HOME=/bdp/hbase/hbase-1.2.1
export PATH=${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:${ZOOKEEPER_HOME}/bin:${HBASE_HOME}/bin:${PATH}
export HADOOP_MAPRED_HOME=${HADOOP_HOME}
export HADOOP_COMMON_HOME=${HADOOP_HOME}
export HADOOP_HDFS_HOME=${HADOOP_HOME}
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP}/lib/native
export YARN_HOME=${HADOOP_HOME}
export HADOOP_OPT="-Djava.library.path=${HADOOP_HOME}/lib/native"
- 配置hbase-site.xml
donaldhan@singleNode:/bdp/hbase/hbase-1.2.1/conf$ cat hbase-site.xml
<configuration>
<property>
<name>hbase.rootdir</name>
<value>/bdp/hbase/data</value>
</property>
<!-- 开启web界面-->
<property>
<name>hbase.master.info.port</name>
<value>60010</value>
</property>
</configuration>
donaldhan@singleNode:/bdp/hbase/hbase-1.2.1/conf$ ^C
- 配置hbase环境变量
donaldhan@singleNode:/bdp/hbase/hbase-1.2.1/conf$ tail -f hbase-env.sh
export JAVA_HOME=/usr/share/jdk1.8.0_191
export HBASE_OPTS="-Xmn150m"
#<!--jvm 年轻代大小,-Xmx，堆最大,-Xms，初始化堆大小-->
#<!-- jvm GC策咯，当年老代使用了80%时回收内存，XX:+UseParNewGC：设置年轻代为并发回收 -XX:+UseConcMarkSweepGC：设置年老代为并发回收-->
export SERVER_GC_OPTS="-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC"
- 启动hbase
start-hbase.sh
直至单机版配置完毕
Hbase客户端
- 引入Hbase JAR 包
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<hbase.version>1.3.0</hbase.version>
</properties>
...
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>${hbase.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>${hbase.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>${hbase.version}</version>
</dependency>
-
将conf下的文件拷贝到项目的resource文件夹下
-
撸客户端代码
package cn.home.hbase;
import cn.home.hbase.entity.User;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* @ClassName: HbaseBoot
* @Description:
* Java版，增强型Hbase API：https://github.com/cnfire/HbaseAPI
* 自定义的spring-boot的hbase starter，为hbase的query和更新等操作提供简易的api并集成spring-boot的auto configuration：
* https://github.com/SpringForAll/spring-boot-starter-hbase
*
* HBase客户端数据管理软件：https://github.com/HY-ZhengWei/HBaseClient
* asynchbase：https://github.com/OpenTSDB/asynchbase
* @Author: Donaldhan
* @Date: 2020-01-30 17:02
*/
@Slf4j
public class HbaseBoot {
private static final String USER_ID_PREFIX = "user-";
private static Admin admin;
public static void main(String[] args){
try {
createTable("user_table", new String[] { "info", "contact" });
User user = new User("001", "xiaoming", "123456", "man", "20", "13355550021", "1232821@csdn.com");
insertData("user_table", user);
User user2 = new User("002", "xiaohong", "654321", "female", "18", "18757912212", "214214@csdn.com");
insertData("user_table", user2);
log.info("--------------------插入两条数据后--------------------");
List<User> list = getAllData("user_table");
list.forEach(e->log.info("after first insert, user:{}",e));
log.info("--------------------获取原始数据-----------------------");
getNoDealData("user_table");
log.info("--------------------根据rowKey查询--------------------");
User user4 = getDataByRowKey("user_table", USER_ID_PREFIX+"001");
log.info("getDataByRowKey 001:{}",user4);
log.info("--------------------获取指定单条数据-------------------");
String user_phone = getCellData("user_table", USER_ID_PREFIX+"001", "contact", "phone");
log.info("getCellData 001 contact phone:{}",user_phone);
User user5 = new User("test-003", "xiaoguang", "789012", "man", "22", "12312132214", "856832@csdn.com");
insertData("user_table", user5);
log.info("--------------------插入测试数据后--------------------");
List<User> list2 = getAllData("user_table");
list2.forEach(e->log.info("after insert test data, user:{}",e));
deleteByRowKey("user_table", "user-test-003");
List<User> list3 = getAllData("user_table");
log.info("--------------------删除测试数据后--------------------");
list3.forEach(e->log.info("after delete test data, user:{}",e));
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 连接集群
* @return
* @throws IOException
*/
public static Connection initHbase() throws IOException {
Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.property.clientPort", "2181");
configuration.set("hbase.zookeeper.quorum", "192.168.5.138");
configuration.setInt("hbase.rpc.timeout",2000);
configuration.setInt("hbase.client.operation.timeout",3000);
configuration.setInt("hbase.client.scanner.timeout.period",6000);
//集群配置
/* configuration.set("hbase.zookeeper.quorum", "192.168.5.135,192.168.5.136,192.168.5.137");
configuration.set("hbase.master", "192.168.5.138:60000");*/
Connection connection = ConnectionFactory.createConnection(configuration);
return connection;
}
/**
* 创建表
* @param tableOriginName
* @param cols
* @throws IOException
*/
public static void createTable(String tableOriginName, String[] cols) throws IOException {
TableName tableName = TableName.valueOf(tableOriginName);
admin = initHbase().getAdmin();
if (admin.tableExists(tableName)) {
// deleteTable(tableNmae);
log.info("{} 表已存在！",tableOriginName);
} else {
HTableDescriptor hTableDescriptor = new HTableDescriptor(tableName);
for (String col : cols) {
HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(col);
hTableDescriptor.addFamily(hColumnDescriptor);
}
admin.createTable(hTableDescriptor);
}
}
/**
* 插入数据
* @param tableName
* @param user
* @throws IOException
*/
public static void insertData(String tableName, User user) throws IOException {
TableName tablename = TableName.valueOf(tableName);
Put put = new Put((USER_ID_PREFIX + user.getId()).getBytes());
//参数：1.列族名 2.列名 3.值
put.addColumn("info".getBytes(), "username".getBytes(), user.getUsername().getBytes()) ;
put.addColumn("info".getBytes(), "age".getBytes(), user.getAge().getBytes()) ;
put.addColumn("info".getBytes(), "gender".getBytes(), user.getGender().getBytes()) ;
put.addColumn("contact".getBytes(), "phone".getBytes(), user.getPhone().getBytes());
put.addColumn("contact".getBytes(), "email".getBytes(), user.getEmail().getBytes());
//HTable table = new HTable(initHbase().getConfiguration(),tablename);已弃用
Table table = initHbase().getTable(tablename);
table.put(put);
}
/**
* 获取原始数据
* @param tableName
*/
public static void getNoDealData(String tableName){
try {
Table table= initHbase().getTable(TableName.valueOf(tableName));
Scan scan = new Scan();
ResultScanner resutScanner = table.getScanner(scan);
for(Result result: resutScanner){
log.info("scan: " + result);
}
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* 根据rowKey进行查询
* @param tableName
* @param rowKey
* @return
* @throws IOException
*/
public static User getDataByRowKey(String tableName, String rowKey) throws IOException {
Table table = initHbase().getTable(TableName.valueOf(tableName));
Get get = new Get(rowKey.getBytes());
User user = new User();
user.setId(rowKey);
//先判断是否有此条数据
if(!get.isCheckExistenceOnly()){
Result result = table.get(get);
for (Cell cell : result.rawCells()){
String colName = Bytes.toString(cell.getQualifierArray(),cell.getQualifierOffset(),cell.getQualifierLength());
String value = Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength());
if(colName.equals("username")){
user.setUsername(value);
}
if(colName.equals("age")){
user.setAge(value);
}
if (colName.equals("gender")){
user.setGender(value);
}
if (colName.equals("phone")){
user.setPhone(value);
}
if (colName.equals("email")){
user.setEmail(value);
}
}
}
return user;
}
/**
* 查询指定单cell内容
* @param tableName
* @param rowKey
* @param family
* @param col
* @return
*/
public static String getCellData(String tableName, String rowKey, String family, String col){
try {
Table table = initHbase().getTable(TableName.valueOf(tableName));
String result = null;
Get get = new Get(rowKey.getBytes());
if(!get.isCheckExistenceOnly()){
get.addColumn(Bytes.toBytes(family),Bytes.toBytes(col));
Result res = table.get(get);
byte[] resByte = res.getValue(Bytes.toBytes(family), Bytes.toBytes(col));
return result = Bytes.toString(resByte);
}else{
return result = "查询结果不存在";
}
} catch (IOException e) {
e.printStackTrace();
}
return "出现异常";
}
/**
* 查询指定表名中所有的数据
* @param tableName
* @return
*/
public static List<User> getAllData(String tableName){
Table table = null;
List<User> list = new ArrayList<User>();
try {
table = initHbase().getTable(TableName.valueOf(tableName));
ResultScanner results = table.getScanner(new Scan());
User user = null;
for (Result result : results){
String id = new String(result.getRow());
log.info("用户名:" + new String(result.getRow()));
user = new User();
for(Cell cell : result.rawCells()){
String row = Bytes.toString(cell.getRowArray(), cell.getRowOffset(), cell.getRowLength());
//String family = Bytes.toString(cell.getFamilyArray(),cell.getFamilyOffset(),cell.getFamilyLength());
String colName = Bytes.toString(cell.getQualifierArray(),cell.getQualifierOffset(),cell.getQualifierLength());
String value = Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength());
user.setId(row);
if(colName.equals("username")){
user.setUsername(value);
}
if(colName.equals("age")){
user.setAge(value);
}
if (colName.equals("gender")){
user.setGender(value);
}
if (colName.equals("phone")){
user.setPhone(value);
}
if (colName.equals("email")){
user.setEmail(value);
}
}
list.add(user);
}
} catch (IOException e) {
e.printStackTrace();
}
return list;
}
/**
* 删除指定cell数据
* @param tableName
* @param rowKey
* @throws IOException
*/
public static void deleteByRowKey(String tableName, String rowKey) throws IOException {
Table table = initHbase().getTable(TableName.valueOf(tableName));
Delete delete = new Delete(Bytes.toBytes(rowKey));
//删除指定列
//delete.addColumns(Bytes.toBytes("contact"), Bytes.toBytes("email"));
table.delete(delete);
}
/**
* 删除表
* @param tableName
*/
public static void deleteTable(String tableName){
try {
TableName tablename = TableName.valueOf(tableName);
admin = initHbase().getAdmin();
admin.disableTable(tablename);
admin.deleteTable(tablename);
} catch (IOException e) {
e.printStackTrace();
}
}
}
实体类
package cn.home.hbase.entity;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* @ClassName: User
* @Description:
* @Author: Donaldhan
* @Date: 2020-01-30 17:16
*/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class User {
private String id;
private String username;
private String password;
private String gender;
private String age;
private String phone;
private String email;
}
启动测试类, 控制台输出
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:java.io.tmpdir=C:\Users\DONALD~1\AppData\Local\Temp\
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:java.compiler=<NA>
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:os.name=Windows 10
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:os.arch=amd64
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:os.version=10.0
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:user.name=Donald_Draper
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:user.home=C:\Users\Donald_Draper
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:100 : Client environment:user.dir=G:\IntellijIdeal\web_test
[WEB_TEST] 2020-01-30 22:12:57 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@7dc19a70
[WEB_TEST] 2020-01-30 22:13:01 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:01 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:01 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb40014, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:04 INFO HBaseAdmin:792 : Created user_table
[WEB_TEST] 2020-01-30 22:13:04 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x32cb636e connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:04 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@63cd604c
[WEB_TEST] 2020-01-30 22:13:04 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:04 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:04 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb40015, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:05 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x12cd9150 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:05 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@114a85c2
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb40016, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:37 : --------------------插入两条数据后--------------------
[WEB_TEST] 2020-01-30 22:13:05 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x54709809 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:05 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@2a2da905
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb40017, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:224 : 用户名:user-001
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:224 : 用户名:user-002
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:39 : after first insert, user:User(id=user-001, username=xiaoming, password=null, gender=man, age=20, phone=13355550021, email=1232821@csdn.com)
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:39 : after first insert, user:User(id=user-002, username=xiaohong, password=null, gender=female, age=18, phone=18757912212, email=214214@csdn.com)
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:40 : --------------------获取原始数据-----------------------
[WEB_TEST] 2020-01-30 22:13:05 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x43ed0ff3 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:05 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@388ffbc2
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb40018, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:135 : scan: keyvalues={user-001/contact:email/1580393585514/Put/vlen=16/seqid=0, user-001/contact:phone/1580393585514/Put/vlen=11/seqid=0, user-001/info:age/1580393585514/Put/vlen=2/seqid=0, user-001/info:gender/1580393585514/Put/vlen=3/seqid=0, user-001/info:username/1580393585514/Put/vlen=8/seqid=0}
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:135 : scan: keyvalues={user-002/contact:email/1580393585761/Put/vlen=15/seqid=0, user-002/contact:phone/1580393585761/Put/vlen=11/seqid=0, user-002/info:age/1580393585761/Put/vlen=2/seqid=0, user-002/info:gender/1580393585761/Put/vlen=6/seqid=0, user-002/info:username/1580393585761/Put/vlen=8/seqid=0}
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:42 : --------------------根据rowKey查询--------------------
[WEB_TEST] 2020-01-30 22:13:05 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x5cc5b667 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:05 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@61edc883
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb40019, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:44 : getDataByRowKey 001:User(id=user-001, username=xiaoming, password=null, gender=man, age=20, phone=13355550021, email=1232821@csdn.com)
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:45 : --------------------获取指定单条数据-------------------
[WEB_TEST] 2020-01-30 22:13:05 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x5f7f2382 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:05 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@407cf41
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:05 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb4001a, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:05 INFO HbaseBoot:47 : getCellData 001 contact phone:13355550021
[WEB_TEST] 2020-01-30 22:13:06 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x5ef8df1e connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:06 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@27cf3151
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb4001b, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:50 : --------------------插入测试数据后--------------------
[WEB_TEST] 2020-01-30 22:13:06 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x411341bd connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:06 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@4c4d362a
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb4001c, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:224 : 用户名:user-001
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:224 : 用户名:user-002
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:224 : 用户名:user-test-003
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:52 : after insert test data, user:User(id=user-001, username=xiaoming, password=null, gender=man, age=20, phone=13355550021, email=1232821@csdn.com)
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:52 : after insert test data, user:User(id=user-002, username=xiaohong, password=null, gender=female, age=18, phone=18757912212, email=214214@csdn.com)
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:52 : after insert test data, user:User(id=user-test-003, username=xiaoguang, password=null, gender=man, age=22, phone=12312132214, email=856832@csdn.com)
[WEB_TEST] 2020-01-30 22:13:06 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x5a021cb9 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:06 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@51768776
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb4001d, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:06 INFO RecoverableZooKeeper:120 : Process identifier=hconnection-0x57f791c6 connecting to ZooKeeper ensemble=192.168.5.138:2181
[WEB_TEST] 2020-01-30 22:13:06 INFO ZooKeeper:438 : Initiating client connection, connectString=192.168.5.138:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@51650883
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:975 : Opening socket connection to server 192.168.5.138/192.168.5.138:2181. Will not attempt to authenticate using SASL (unknown error)
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:852 : Socket connection established to 192.168.5.138/192.168.5.138:2181, initiating session
[WEB_TEST] 2020-01-30 22:13:06 INFO ClientCnxn:1235 : Session establishment complete on server 192.168.5.138/192.168.5.138:2181, sessionid = 0x16ff6c2fcb4001e, negotiated timeout = 40000
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:224 : 用户名:user-001
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:224 : 用户名:user-002
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:55 : --------------------删除测试数据后--------------------
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:56 : after delete test data, user:User(id=user-001, username=xiaoming, password=null, gender=man, age=20, phone=13355550021, email=1232821@csdn.com)
[WEB_TEST] 2020-01-30 22:13:06 INFO HbaseBoot:56 : after delete test data, user:User(id=user-002, username=xiaohong, password=null, gender=female, age=18, phone=18757912212, email=214214@csdn.com)
小节
上面只是Hbase简单客户单的使用，如果在生产环境还无法使用，更多增强客户端和桌面客户端见，及与SpringBoot的集成参见如下链接：
- Java版，增强型Hbase API：https://github.com/cnfire/HbaseAPI
- 自定义的spring-boot的hbase starter，为hbase的query和更新等操作提供简易的api并集成spring-boot的auto configuration： https://github.com/SpringForAll/spring-boot-starter-hbase
- HBase客户端数据管理软件：https://github.com/HY-ZhengWei/HBaseClient
- asynchbase：https://github.com/OpenTSDB/asynchbase
附
在测试的过程中，出现问题如下：
- org.apache.hadoop.hbase.DoNotRetryIOException
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:229)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:202)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:326)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:301)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:166)
at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:161)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:794)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:403)
at cn.home.hbase.HbaseBoot.createTable(HbaseBoot.java:90)
at cn.home.hbase.HbaseBoot.main(HbaseBoot.java:32)
Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:601)
at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:585)
at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:564)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
... 10 more
Disconnected from the target VM, address: '127.0.0.1:54688', transport: 'socket'
Stopwatch在google的guava包下，由于从guava17开始，constructors发生变化，如果确认当前项目只有一个guava包，需确认版本问题。否则解决guava的冲突
测试，hbase1.2.1：guava12-16，运行无误；guava17开始，同样出现标题异常；
如果想偷懒，可以升级客户端版本1.3.0
- org.apache.hadoop.hbase.client.RetriesExhaustedException
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
Thu Jan 30 20:44:58 GMT+08:00 2020, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=63294: ubuntu row 'user_table,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=ubuntu,41874,1580387982919, seqNum=0
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:276)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:210)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:162)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:796)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:408)
at cn.home.hbase.HbaseBoot.createTable(HbaseBoot.java:89)
at cn.home.hbase.HbaseBoot.main(HbaseBoot.java:32)
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=63294: ubuntu row 'user_table,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=ubuntu,41874,1580387982919, seqNum=0
2020-01-30 20:55:58,977 INFO [ubuntu,45279,1580387980864_splitLogManager__ChoreService_1] hbase.ScheduledChore: Chore: SplitLogManager Timeout Monitor missed its start time
这个问题的关键点为：
2020-01-30 20:55:58,977 INFO [ubuntu,45279,1580387980864_splitLogManager__ChoreService_1] hbase.ScheduledChore: Chore: SplitLogManager Timeout Monitor missed its start time
即找不到ubuntu机器；为了解决这个问题，修改hosts
donaldhan@ubuntu:/bdp/hbase/data/data/default$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 ubuntu
192.168.5.138 singleNode
# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
donaldhan@ubuntu:/bdp/hbase/data/data/default$ sudo vim /etc/hosts
[sudo] password for donaldhan:
donaldhan@ubuntu:/bdp/hbase/data/data/default$ cat /etc/hosts
127.0.0.1 localhost
127.0.0.1 ubuntu
192.168.5.138 singleNode
# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
donaldhan@ubuntu:/bdp/hbase/data/data/default$
没有效果，修改如下
donaldhan@ubuntu:/bdp/hbase/data/data/default$ cat /etc/hosts
127.0.0.1 localhost
192.168.5.138 singleNode
# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
donaldhan@ubuntu:/bdp/hbase/data/data/default$
又包相同的错误，只不过主机名不一样
Thu Jan 30 21:18:22 GMT+08:00 2020, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=77067:
Connection refused: no further information row 'user_table,,'
on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=localhost,35007,1580389625985, seqNum=0
难regionservers配置有问题，单机的不应该呀？，修改regionServer，重启
donaldhan@ubuntu:/bdp/hbase/hbase-1.2.1/conf$ cat regionservers
singleNode
donaldhan@ubuntu:/bdp/hbase/data/data/default$ cat /etc/hostname
singleNode
无效，有人说给出了如下解决方式：
windows下开发HBase应用程序，HBase部署在linux环境中，
在运行调试时可能会出现无法找到主机，类似异常信息如下：
java.net.UnknownHostException: unknown host: master
解决办法如下：
在C:\WINDOWS\system32\drivers\etc\hosts文件中添加如下信息：
192.0.0.1 master
已经配置，仍然无效
# VmHost
192.168.5.130 pseduoDisHadoop
192.168.5.135 nameNode
192.168.5.136 secondlyNameNode
192.168.5.137 resourceManager
192.168.5.138 singleNode
192.168.5.138 localhost
突然灵感闪现，是不是hostname的问题，修改hostname
donaldhan@singleNode:~$ cat /etc/hostname
singleNode
重启机器，重启Hbase，运营测试类，测试成功。
再次观察 http://singlenode:60010/master-status Hbase Web监控界面
Master singleNode
Region Servers
Base Stats Memory Requests Storefiles Compactions
ServerName Start time Version Requests Per Second Num. Regions
singlenode,38834,1580392979033 Thu Jan 30 22:02:59 CST 2020 1.2.1 0 3
Total:1 0 3
Dead Region Servers
ServerName Stop time
localhost,33294,1580390860209 Thu Jan 30 22:03:08 CST 2020
localhost,34594,1580392459858 Thu Jan 30 22:03:08 CST 2020
Master为singleNode， Region Servers为localhost，证明不是regionServer的原因