PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)

PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)

目录

前言--Thrift1和Thrift2的简要介绍

0.1 写在前面

  从Hbase0.94.11开始有两套thrift接口(可以叫Thrift1和Thrift2),根据官方文档,Thrift1很可能被抛弃,但网上的文章基本是介绍Thrift1的,本文则兼容介绍Thrift1和Thrift2,目前在网上是比较详细的介绍Thrift2文章了。
  Thrift2相比较Thrift1做了简化和合成,但不提供查询所有Table、创建Table、删除Table的功能了。日常在工作中应该也很少会在Hbase中使用Thrift来创建Table。
  从Thrift0.9.1开始支持golang语言,相关使用和demo可以参考这里《Golang、Php、Python、Java基于Thrift0.9.1实现跨语言调用》,不过由于写本文时Thrift0.9.1中对Golang访问Hbase支持的并不友好,在0.9.2中的Thrift2已经进行了改进可以很好的支持Golang访问Hbase。所以后面在使用Golang访问Hbase时,使用的是Thrift0.9.2。
  网上关于Thrift1的文章一搜索有很多,也就不在这里多描述,在后面的演示中Thrift2中90%的方法都会有,不过在实际使用中可能还需要根据你的系统要求进行二次封装 。
  

0.2 Thrift1和Thrift2的区别

  现在官网也没有太详细的介绍,下面的对比也是从IDL描述中找到的。这是一位网友整理的,后面的扩展阅读中链接了他的文章,更多的介绍可以去那里看。

 

Thrift

Thrift2

结构 struct TCell
struct ColumnDescriptor
struct TRegionInfo
struct Mutation
struct BatchMutation
struct TIncrement
struct TColumn
struct TRowResult
struct TScan
struct TTimeRange
struct TColumn
struct TColumnValue
struct TColumnIncrement
struct TResult
struct TGet
struct TPut
struct TDelete
struct TIncrement
struct TScan
struct TRowMutations
异常 exception IOError
exception IllegalArgument
exception AlreadyExists
exception TIOError
exception TIllegalArgument
其他   union TMutation
enum TDeleteType
enum TDurability
服务 名称为:Hbase
void enableTable()
void disableTable()
bool isTableEnabled()
void compact()
void majorCompact()
list<Text> getTableNames()
map<Text,ColumnDescriptor> getColumnDescriptors()
list<TRegionInfo> getTableRegions()
void createTable()
void deleteTable()
list<TCell> get()
list<TCell> getVer()
list<TCell> getVerTs()
list<TRowResult> getRow()
list<TRowResult> getRowWithColumns()
list<TRowResult> getRowTs()
list<TRowResult> getRowWithColumnsTs()
list<TRowResult> getRows()
list<TRowResult> getRowsWithColumns()
list<TRowResult> getRowsTs()
list<TRowResult> getRowsWithColumnsTs()
void mutateRow()
void mutateRowTs()
void mutateRows()
void mutateRowsTs()
i64 atomicIncrement()
void deleteAll()
void deleteAllTs()
void deleteAllRow()
void increment()
void incrementRows()
void deleteAllRowTs()
ScannerID scannerOpenWithScan()
ScannerID scannerOpen()
ScannerID scannerOpenWithStop()
ScannerID scannerOpenWithPrefix()
ScannerID scannerOpenTs()
ScannerID scannerOpenWithStopTs()
list<TRowResult> scannerGet()
list<TRowResult> scannerGetList()
void scannerClose()
list<TCell> getRowOrBefore()
TRegionInfo getRegionInfo()
名称为:THBaseService
bool exists(...)
TResult get(...)
list<TResult> getMultiple(...)
void put(...)
bool checkAndPut(...)
void putMultiple(...)
void deleteSingle(...)
list<TDelete> deleteMultiple(...)
bool checkAndDelete(...)
TResult increment(...)
i32 openScanner(...)
list<TResult> getScannerRows(...)
void closeScanner(...)
void mutateRow(...)
list<TResult> getScannerResults(...)

1. Thrift0.9.2的安装

1.1 安装依赖插件

root@m1:/home/hadoop#sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
#下载thirft0.9.2版本
root@m1:/home/hadoop# git clone https://github.com/apache/thrift.git thrift-git0.9.2
Cloning into 'thrift-git0.9.2'...
remote: Counting objects: 37221, done.
remote: Compressing objects: 100% (195/195), done.
remote: Total 37221 (delta 312), reused 407 (delta 272)
Receiving objects: 100% (37221/37221), 9.61 MiB | 74 KiB/s, done.
Resolving deltas: 100% (25830/25830), done.
root@m1:/home/hadoop#cd thrift-git0.9.2
# 切换成相应所要用的分支,如果你编译通过,就不用像我现在这样切换了
root@m2:/home/hadoop/thrift-git0.9.2# git checkout --track origin/0.9.2
Branch 0.9.2 set up to track remote branch 0.9.2 from origin.
Switched to a new branch ‘0.9.2’

1.2 Thrift0.9.2的编译

  编译Thrift0.9.2版本,注意这个版本和0.9.1编译方法不一样,详情可以参考/home/hadoop/thrift-git0.9.2/compiler/cpp/README.md中的说明

root@m1:/home/hadoop/thrift-git0.9.2# mkdir -p compiler/cpp/build
root@m1:/home/hadoop/thrift-git0.9.2# cd compiler/cpp/build
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# cmake ..
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Thrift package version: 0.9.2
-- Thrift version: 0.9.2 (0.9.2)
-- Found FLEX: /usr/bin/flex (found version "2.5.35")
-- Found BISON: /usr/bin/bison (found version "2.5")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/hadoop/thrift-git0.9.2/compiler/cpp/build
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# make
[  3%] [BISON][thrifty] Building parser with bison 2.5
[  6%] Generating thrifty.h
[  9%] [FLEX][thriftl] Building scanner with flex 2.5.35
Scanning dependencies of target libparse
[ 12%] Building CXX object CMakeFiles/libparse.dir/thrifty.cc.o
[ 15%] Building CXX object CMakeFiles/libparse.dir/thriftl.cc.o
Linking CXX static library liblibparse.a
[ 15%] Built target libparse
Scanning dependencies of target thrift
[ 18%] Building CXX object CMakeFiles/thrift.dir/src/main.cc.o
[ 21%] Building C object CMakeFiles/thrift.dir/src/md5.c.o
[ 25%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_generator.cc.o
[ 28%] Building CXX object CMakeFiles/thrift.dir/src/parse/t_typedef.cc.o
[ 31%] Building CXX object CMakeFiles/thrift.dir/src/parse/parse.cc.o
[ 34%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_c_glib_generator.cc.o
[ 37%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_cpp_generator.cc.o
[ 40%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_java_generator.cc.o
[ 43%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_as3_generator.cc.o
[ 46%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_csharp_generator.cc.o
[ 50%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_py_generator.cc.o
[ 53%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_rb_generator.cc.o
[ 56%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_perl_generator.cc.o
[ 59%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_php_generator.cc.o
[ 62%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_erl_generator.cc.o
[ 65%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_cocoa_generator.cc.o
[ 68%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_st_generator.cc.o
[ 71%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_ocaml_generator.cc.o
[ 75%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_hs_generator.cc.o
[ 78%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_xsd_generator.cc.o
[ 81%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_html_generator.cc.o
[ 84%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_js_generator.cc.o
[ 87%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_javame_generator.cc.o
[ 90%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_delphi_generator.cc.o
[ 93%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_go_generator.cc.o
[ 96%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_d_generator.cc.o
[100%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_lua_generator.cc.o
Linking CXX executable thrift
[100%] Built target thrift
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build#
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# ./thrift -version
Thrift version 0.9.2

1.3 Thrift0.9.2编译后,配置成可执行文件

  将编译后的Thrift0.9.2放到可执行程序目录,并命名为thrift092

root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# whereis thrift
thrift: /usr/local/bin/thrift
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# cp thrift /usr/local/bin/thrift092
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# thrift092 -version
Thrift version 0.9.2
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build#

2. Hbase0.96.2的安装

2.1 Hbase的介绍

  hbase的介绍,参考这里:http://baike.baidu.com/view/1993870.htm

2.2 Hbase的安装

  请参考这篇文章:《ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1伪分布式环境部署
  后面的测试,也是基于上面这篇文章的环境测试的。

2.3 Hbase基于源码的Thrift生成接口文件

  下载Hbase的源码包(后面会使用里面的Thrift1和Thrift2)

root@m1:/home/hadoop# wget http://mirrors.hust.edu.cn/apache/hbase/hbase-0.96.2/hbase-0.96.2-src.tar.gz
root@m1:/home/hadoop# tar xzvf hbase-0.96.2-src.tar.gz

  #创建测试目录

root@m1:/home/hadoop# mkdir thrift_hbase
root@m1:/home/hadoop# cd thrift_hbase/

  生成Php基于Hbase的Thrift1的接口文件,由于Thrift0.9.1版本对Golang访问Hbase支持不够后,后面Golang访问Hbase用Thrift0.9.2来测试

root@m1:/home/hadoop/thrift_hbase# thrift --gen php /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
root@m1:/home/hadoop/thrift_hbase# ll
总用量 16
drwxr-xr-x  4 root   root   4096 Aug 19 21:19 ./
drwxr-xr-x 47 hadoop hadoop 4096 Aug 19 21:15 ../
drwxr-xr-x  3 root   root   4096 Aug 19 21:19 gen-php/
root@m1:/home/hadoop/thrift_hbase#

3. PHP和Golang使用Thrift1和Thrift2访问Hbase

3.1 PHP基于Thrift1访问Hbase

  将Php用到的Thrift1包复制到thrift_hbase里面

root@m1:/home/hadoop/thrift_hbase# mkdir libphp                      
root@m1:/home/hadoop/thrift_hbase# cp -r /home/hadoop/thrift-git/lib/php/src/* libphp/
root@m1:/home/hadoop/thrift_hbase# cp -r /home/hadoop/thrift-git/lib/php/lib/Thrift libphp/
root@m1:/home/hadoop/thrift_hbase# cp -r gen-php/Hbase libphp/
root@m1:/home/hadoop/thrift_hbase# rm -rf gen-php/

  #编写Hbase.php文件

<?php 
/**
 * PHP基于Hbase0.96.2的Thrift1访问Hbase
 *
 * @author     迦壹
 * @copyright  Copyright (c) 2014 (http://idoall.org)
 * @license    GNU General Public License 2.0
 * @version    1
 */
 $startTime = getMillisecond();
$GLOBALS['THRIFT_ROOT'] = './libphp';   # 指定库目录,可以是绝对路径或是相对路径 
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/ClassLoader/ThriftClassLoader.php';

use Thrift\ClassLoader\ThriftClassLoader; 
use Thrift\Type\TMessageType;
use Thrift\Type\TType; 
use Thrift\Exception\TException;
use Thrift\Factory\TStringFuncFactory;
use Thrift\StringFunc\TStringFunc;
use Thrift\StringFunc\Core;
use Thrift\Protocol\TBinaryProtocol; 
use Thrift\Transport\TSocket; 
use Thrift\Transport\TSocketPool; 
use Thrift\Transport\TBufferedTransport;
use Hbase\HbaseClient;
use Hbase\ColumnDescriptor;
use Hbase\Mutation;

$loader = new ThriftClassLoader(); 
$loader->registerNamespace('Thrift', $GLOBALS['THRIFT_ROOT']); # 加载thrift 
$loader->register();

require_once( $GLOBALS['THRIFT_ROOT'].'/Hbase/Hbase.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/Hbase/Types.php' );

$socket = new TSocket( 'm1', 9090 );
$socket->setSendTimeout( 10000 ); // Ten seconds (too long for production, but this is just a demo 😉
$socket->setRecvTimeout( 20000 ); // Twenty seconds
$transport = new TBufferedTransport( $socket );
$protocol = new TBinaryProtocol( $transport );
$client = new HbaseClient( $protocol );

$transport->open();
$tmpendTime = getMillisecond();
echo "----建立连接用时:".$tmpendTime."-".$startTime."=".($tmpendTime-$startTime)."毫秒\n";

//测试表的名称
$tempTableName = "idoallorg_student";
$tmpstartTime = getMillisecond();
$tables = $client->getTableNames();
sort($tables);
echo "----列出所有的Table:----\n";
foreach ( $tables as $name ) {
  echo( "\tfound: {$name}\n" );
  if ( $name == $tempTableName ) {
    if ($client->isTableEnabled( $tempTableName )) {
      echo( "\t表已经存在,即将关闭: {$tempTableName}\n");
      $client->disableTable( $tempTableName );
    }
    echo( "\t删除表: {$tempTableName}\n" );
    $client->deleteTable( $tempTableName );
  }
}
$tmpendTime = getMillisecond();
echo "----列出所有Table用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";



echo( "----创建表: {$tempTableName}----\n" );
$tmpstartTime = getMillisecond();
$columns = array(
  new ColumnDescriptor( array(
    'name' => 'id',
    'maxVersions' => 10
  ) ),
  new ColumnDescriptor( array(
    'name' => 'name:'
  ) ),
  new ColumnDescriptor( array(
    'name' => 'score:'
  ) )
);
try {
  $client->createTable( $tempTableName, $columns );
} catch ( AlreadyExists $ae ) {
  echo( "WARN: {$ae->message}\n" );
}
$tmpendTime = getMillisecond();
echo "----创建表: {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

//读取创建后的表结构
//$tmpstartTime = getMillisecond();
echo( "----查看表结构: {$tempTableName}:----\n" );
$tmpstartTime = getMillisecond();
$descriptors = $client->getColumnDescriptors( $tempTableName );
asort( $descriptors );
foreach ( $descriptors as $col ) {
  echo( "\tcolumn: {$col->name}, maxVer: {$col->maxVersions}\n" );
}

$tmpendTime = getMillisecond();
echo "----查看表结构: {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";

//
echo "----向表{$tempTableName} 中写入1条数据----\n";
$tmpstartTime = getMillisecond();
$row = '1';
$valid = "foobar-000000";
$valid1 = "foobar-11111111";

$mutations = array(
    new Mutation(array(
            'column' => 'id:',
            'value' => 1,
        )),
    new Mutation(array(
            'column' => 'score:a',
            'value' => $valid,
        )),
    new Mutation(array(
            'column' => 'name:a',
            'value' => "idoall.org",
        )),
);
$client->mutateRow($tempTableName, $row, $mutations, null);
$tmpendTime = getMillisecond();
echo "----向表{$tempTableName} 中写入1条数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";

//
echo "----获取刚写入的score列的数据->get----\n";
$tmpstartTime = getMillisecond();
$arr = $client->get($tempTableName, $row, "score:a", null);
foreach ($arr as $k => $v) {
    // $k = TCell
    echo "\t ------ get one : value = {$v->value}\n";
    echo "\t ------ get one : timestamp = {$v->timestamp}\n";
}
$tmpendTime = getMillisecond();
echo "----获取刚写入的 score列的数据->get用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";

//
echo "----获取刚写入的数据->getRow----\n";
$tmpstartTime = getMillisecond();
$result = $client->getRow($tempTableName, $row, null);
foreach ($result as $key => $TRowResult) {
    echo "\trowkey:$TRowResult->row\tcolumns(array):\r\n";
    foreach ($TRowResult->columns as $key => $value) {
        echo "\t\t\tkey:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";
    }
}
$tmpendTime = getMillisecond();
echo "----获取刚写入的数据->getRow用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";


//
echo "----测试更新数据----\n";
$tmpstartTime = getMillisecond();
try {
  $mutations = array(
    new Mutation( array(
      'column' => 'score:a',
      'value' => $valid1
    ) ),
  );
  $client->mutateRow( $tempTableName, $row, $mutations, null );
} catch ( IOError $e ) {
  echo( "expected error: {$e->message}\n" );
}
$tmpendTime = getMillisecond();
echo "----测试更新数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";


echo "----批量更新10条数据----\n";
$tmpstartTime = getMillisecond();
$mutationsbatch = array();
for($i=0;$i<10;$i++)
{
    $rowkey=$i+10;
    $mutations = array(
        new Mutation(array(
                'column' => 'id:int',
                'value' => $i+10,
            )),
        new Mutation(array(
                'column' => 'score:txt',
                'value' => "score".$i,
            )),
        new Mutation(array(
                'column' => 'name:txt',
                'value' => "idoall.org".$i,
            )),
    );
    array_push($mutationsbatch, new \Hbase\BatchMutation(array('row'=>$rowkey, 'mutations'=>$mutations)));
}
$client->mutateRows($tempTableName, $mutationsbatch,null);
$tmpendTime = getMillisecond();
echo "----批量更新10条数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";


echo "----scanner get scannerOpen-----\r\n";
$tmpstartTime = getMillisecond();
$startRow = 0;
$scan = $client->scannerOpen($tempTableName, $startRow, array ('column' => 'score', 'column'=>'name'), null);
$nbRows = 20;
$arr = $client->scannerGetList($scan, $nbRows);
echo 'count of result :'.count($arr)."\n";
foreach ($arr as $k => $TRowResult) {
    echo "\trow:$TRowResult->row\tcolumns(array):";
    foreach ($TRowResult->columns as $key => $value) {
        echo "key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";
    }
}
$client->scannerClose($scan);
$tmpendTime = getMillisecond();
echo "----scanner get scannerOpen用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";


echo "----通过RowKey的范围获取数据 scannerOpenWithStop-----\r\n";
$tmpstartTime = getMillisecond();
$result = $client->scannerOpenWithStop($tempTableName , $startRow, $nbRows, array ('column' => 'score', ),null);
$recordArray = array();
while (true) {
    $record = $client->scannerGet($result);
    if ($record == NULL) {
        break;
    }
    foreach($record as $TRowResult) {
        echo "\trow:$TRowResult->row\tcolumns(array):";
        foreach ($TRowResult->columns as $key => $value) {
            echo "key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";
        }
    }
}

$tmpendTime = getMillisecond();
echo "----通过RowKey的范围获取数据 scannerOpenWithStop用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";


//删除表
echo "----删除表 {$tempTableName}----\n";
$tmpstartTime = getMillisecond();
try {
    $tmpstartTime_disableTable = getMillisecond();
    $client->disableTable( $tempTableName );
    $tmpendTime_disableTable = getMillisecond();
    echo "\t其中关闭表 {$tempTableName}用时:".$tmpendTime_disableTable."-".$tmpstartTime_disableTable."=".($tmpendTime_disableTable-$tmpstartTime_disableTable)."毫秒\n";
    $tmpstartTime_disableTable = getMillisecond();
    $client->deleteTable($tempTableName);
    $tmpendTime_disableTable = getMillisecond();
    echo "\t其中删除表 {$tempTableName}用时:".$tmpendTime_disableTable."-".$tmpstartTime_disableTable."=".($tmpendTime_disableTable-$tmpstartTime_disableTable)."毫秒\n";
} catch ( AlreadyExists $ae ) {
  echo( "WARN: {$ae->message}\n" );
}
$tmpendTime = getMillisecond();
echo "----删除表 {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

$endTime = getMillisecond();

 echo "\r\n";
echo "PHP调用 总计 用时:".$endTime."-".$startTime."=".($endTime-$startTime)."毫秒\n";

function getMillisecond() {
list($t1, $t2) = explode(' ', microtime());     
return (float)sprintf('%.0f', (floatval($t1) + floatval($t2)) * 1000);  
}

$transport->close();
?>

  启动Hbase的Thrift,如果不了解,看下这篇文章吧,本文的环境都是基于这篇文章的《ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1伪分布式环境部署

root@m1:/home/hadoop/thrift_hbase# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift

  运行Hbase.php,如果你也可以看到下面和我一样的提示,那么恭喜你也成功地使用PHP通过Thrift1访问到Hbase
  

root@m1:/home/hadoop/thrift_hbase# php hbase.php 
----建立连接用时:1408545184136-1408545184102=34毫秒
----列出所有的Table:----
        found: demo_table
        found: hbase2hive_idoall
        found: hive2hbase_idoall
        found: test_idoall_org
----列出所有Table用时:1408545184229-1408545184136=93毫秒

----创建表: idoallorg_student----
----创建表: idoallorg_student用时:1408545185026-1408545184229=797毫秒
----查看表结构: idoallorg_student:----
        column: id:, maxVer: 10
        column: name:, maxVer: 3
        column: score:, maxVer: 3
----查看表结构: idoallorg_student用时:1408545185047-1408545185026=21毫秒

----向表idoallorg_student 中写入1条数据----
----向表idoallorg_student 中写入1条数据用时:1408545185288-1408545185047=241毫秒

----获取刚写入的score列的数据->get----
         ------ get one : value = foobar-000000
         ------ get one : timestamp = 1408545184901
----获取刚写入的 score列的数据->get用时:1408545185373-1408545185289=84毫秒

----获取刚写入的数据->getRow----
        rowkey:1        columns(array):
                        key:id: value:1 timestamp:1408545184901
                        key:name:a      value:idoall.org        timestamp:1408545184901
                        key:score:a     value:foobar-000000     timestamp:1408545184901
----获取刚写入的数据->getRow用时:1408545185383-1408545185373=10毫秒

----测试更新数据----
----测试更新数据用时:1408545185389-1408545185384=5毫秒

----批量更新10条数据----
----批量更新10条数据用时:1408545185442-1408545185389=53毫秒

----scanner get scannerOpen-----
count of result :11
        row:1   columns(array):key:name:a       value:idoall.org        timestamp:1408545184901
        row:10  columns(array):key:name:txt     value:idoall.org0       timestamp:1408545185053
        row:11  columns(array):key:name:txt     value:idoall.org1       timestamp:1408545185053
        row:12  columns(array):key:name:txt     value:idoall.org2       timestamp:1408545185053
        row:13  columns(array):key:name:txt     value:idoall.org3       timestamp:1408545185053
        row:14  columns(array):key:name:txt     value:idoall.org4       timestamp:1408545185053
        row:15  columns(array):key:name:txt     value:idoall.org5       timestamp:1408545185053
        row:16  columns(array):key:name:txt     value:idoall.org6       timestamp:1408545185053
        row:17  columns(array):key:name:txt     value:idoall.org7       timestamp:1408545185053
        row:18  columns(array):key:name:txt     value:idoall.org8       timestamp:1408545185053
        row:19  columns(array):key:name:txt     value:idoall.org9       timestamp:1408545185053
----scanner get scannerOpen用时:1408545185484-1408545185443=41毫秒

----通过RowKey的范围获取数据 scannerOpenWithStop-----
        row:1   columns(array):key:score:a      value:foobar-11111111   timestamp:1408545185003
        row:10  columns(array):key:score:txt    value:score0    timestamp:1408545185053
        row:11  columns(array):key:score:txt    value:score1    timestamp:1408545185053
        row:12  columns(array):key:score:txt    value:score2    timestamp:1408545185053
        row:13  columns(array):key:score:txt    value:score3    timestamp:1408545185053
        row:14  columns(array):key:score:txt    value:score4    timestamp:1408545185053
        row:15  columns(array):key:score:txt    value:score5    timestamp:1408545185053
        row:16  columns(array):key:score:txt    value:score6    timestamp:1408545185053
        row:17  columns(array):key:score:txt    value:score7    timestamp:1408545185053
        row:18  columns(array):key:score:txt    value:score8    timestamp:1408545185053
        row:19  columns(array):key:score:txt    value:score9    timestamp:1408545185053
----通过RowKey的范围获取数据 scannerOpenWithStop用时:1408545185536-1408545185484=52毫秒

----删除表 idoallorg_student----
        其中关闭表 idoallorg_student用时:1408545186819-1408545185536=1283毫秒
        其中删除表 idoallorg_student用时:1408545186974-1408545186819=155毫秒
----删除表 idoallorg_student用时:1408545186974-1408545185536=1438毫秒

本次调用 总计 用时:1408545186974-1408545184102=2872毫秒

3.2 Golang基于Thrift2访问Hbase

  生成Golang基于Hbase的Thrift2的接口文件

root@m1:/home/hadoop/thrift_hbase# thrift092 -r --gen go /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift2/hbase.thrift

  将Thrift2生成的开发库复制到GOPATH的src中
  删除掉0.9.1的文件,可能作者比较匆忙,在我使用0.9.2生成Thrift2的时候,还存在Thrift1的接口,这会对我们的测试带来干扰(后来我下载thrift1.0.0 dev时发现已经删除了这个文件,如果你生成IDL后的接口文件中,不包含这个文件,可以不用删除

root@m1:/home/hadoop/thrift_hbase# rm gen-go/hbase/t_h_base_service.go
root@m1:/home/hadoop/thrift_hbase# cp -r gen-go/hbase $GOPATH/src

  编写hbase.go文件

//
//Golang基于Hbase0.96.2的Thrift2访问Hbase
//
//@author     迦壹
//@copyright  Copyright (c) 2014 (http://idoall.org)
//@license    GNU General Public License 2.0
//@version    1

package main

import (
    "encoding/binary"
    "fmt"
    "git.apache.org/thrift.git/lib/go/thrift"
    "hbase"
    "net"
    "os"
    "reflect"
    "strconv"
    "time"
)

const (
    HOST       = "m1"
    PORT       = "9090"
    TESTRECORD = 10
)

func main() {
    startTime := currentTimeMillis()
    logformatstr_ := "----%s\n"
    logformatstr := "----%s 用时:%d-%d=%d毫秒\n\n"
    logformattitle := "建立连接"
    rowkey := "1"
    temptable := "test_idoall_org"

    protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()

    transport, err := thrift.NewTSocket(net.JoinHostPort(HOST, PORT))
    if err != nil {
        fmt.Fprintln(os.Stderr, "error resolving address:", err)
        os.Exit(1)
    }

    client := hbase.NewTHBaseServiceClientFactory(transport, protocolFactory)
    if err := transport.Open(); err != nil {
        fmt.Fprintln(os.Stderr, "Error opening socket to "+HOST+":"+PORT, " ", err)
        os.Exit(1)
    }
    tmpendTime := currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, startTime, (tmpendTime - startTime))
    defer transport.Close()

    //--------------Exists
    logformattitle = "调用Exists方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime := currentTimeMillis()
    //
    isexists, err := (client.Exists([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
    fmt.Printf("rowkey{%s} in table{%s} Exists:%t\t", rowkey, temptable, isexists)
    if err != nil {
        fmt.Printf("Exists err:%s\n", err)
    }
    fmt.Println("")
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //--------------put
    logformattitle = "调用Put方法写数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    cvarr := []*hbase.TColumnValue{
        &hbase.TColumnValue{
            Family:    []byte("name"),
            Qualifier: []byte("idoall.org"),
            Value:     []byte("welcome idoall.org")}}
    temptput := hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}
    err = client.Put([]byte(temptable), &temptput)
    if err != nil {
        fmt.Printf("Put err:%s\n", err)
    } else {
        fmt.Println("Put done")
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------Get---------------
    logformattitle = "调用Get方法获取新增加的数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    //
    result, err := (client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
    if err != nil {
        fmt.Printf("Get err:%s\n", err)
    } else {
        fmt.Println("Rowkey:" + string(result.Row))
        for _, cv := range result.ColumnValues {
            printscruct(cv)
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //--------------put update
    logformattitle = "调用Put update方法'修改'数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    cvarr = []*hbase.TColumnValue{
        &hbase.TColumnValue{
            Family:    []byte("name"),
            Qualifier: []byte("idoall.org"),
            Value:     []byte("welcome idoall.org---update")}}
    temptput = hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}
    err = client.Put([]byte(temptable), &temptput)
    if err != nil {
        fmt.Printf("Put update err:%s\n", err)
    } else {
        fmt.Println("Put update done")
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------Get  update---------------
    logformattitle = "调用Get方法获取'修改'后的数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    //
    result, err = (client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
    if err != nil {
        fmt.Printf("Get update err:%s\n", err)
    } else {
        fmt.Println("update Rowkey:" + string(result.Row))
        for _, cv := range result.ColumnValues {
            printscruct(cv)
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------DeleteSingle------------
    logformattitle = "调用DeleteSingle方法删除一条数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    tdelete := hbase.TDelete{Row: []byte(rowkey)}
    err = client.DeleteSingle([]byte(temptable), &tdelete)
    if err != nil {
        fmt.Printf("DeleteSingle err:%s\n", err)
    } else {
        fmt.Printf("DeleteSingel done\n")
    }

    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //-------------PutMultiple----------------
    logformattitle = "调用PutMultiple方法添加" + strconv.Itoa(TESTRECORD) + "条数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()

    var tputArr []*hbase.TPut
    for i := 0; i < TESTRECORD; i++ {
        putrowkey := strconv.Itoa(i)

        tputArr = append(tputArr, &hbase.TPut{
            Row: []byte(putrowkey),
            ColumnValues: []*hbase.TColumnValue{
                &hbase.TColumnValue{
                    Family:    []byte("name"),
                    Qualifier: []byte("idoall.org"),
                    Value:     []byte(time.Now().String())}}})
    }

    err = client.PutMultiple([]byte(temptable), tputArr)
    if err != nil {
        fmt.Printf("PutMultiple err:%s\n", err)
    } else {
        fmt.Printf("PutMultiple done\n")
    }

    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------------GetMultiple-----------------------------
    logformattitle = "调用GetMultiple方法获取" + strconv.Itoa(TESTRECORD) + "数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    //
    var tgets []*hbase.TGet
    for i := 0; i < TESTRECORD; i++ {
        putrowkey := strconv.Itoa(i)
        tgets = append(tgets, &hbase.TGet{
            Row: []byte(putrowkey)})
    }
    results, err := client.GetMultiple([]byte(temptable), tgets)
    if err != nil {
        fmt.Printf("GetMultiple err:%s", err)
    } else {
        fmt.Printf("GetMultiple Count:%d\n", len(results))
        for _, k := range results {
            fmt.Println("Rowkey:" + string(k.Row))
            for _, cv := range k.ColumnValues {
                printscruct(cv)
            }
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
    //-------------------TMutation
    //TMutation包含一个TGet一个TPut,就不做测试了
    //可以和MutateRow结合使用
    //

    //-------------------OpenScanner
    logformattitle = "调用OpenScanner方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    startrow := make([]byte, 4)
    binary.LittleEndian.PutUint32(startrow, 1)
    stoprow := make([]byte, 4)
    binary.LittleEndian.PutUint32(stoprow, 10)

    scanresultnum, err := client.OpenScanner([]byte(temptable), &hbase.TScan{
        StartRow: startrow,
        StopRow:  stoprow,
        //     FilterString: []byte("RowFilter(=, 'regexstring:00[1-3]00')"),
        //     FilterString: []byte("PrefixFilter('1407658495588-')"),
        Columns: []*hbase.TColumn{
            &hbase.TColumn{
                Family:    []byte("name"),
                Qualifier: []byte("idoall.org")}}})
    if err != nil {
        fmt.Printf("OpenScanner err:%s\n", err)
    } else {
        fmt.Printf("OpenScanner %d done\n", scanresultnum)

        scanresult, err := client.GetScannerRows(scanresultnum, 100)
        if err != nil {
            fmt.Printf("GetScannerRows err:%s\n", err)
        } else {
            fmt.Printf("GetScannerRows %d done\n", len(scanresult))
            for _, k := range scanresult {
                fmt.Println("scan Rowkey:" + string(k.Row))
                for _, cv := range k.ColumnValues {
                    printscruct(cv)
                }
            }
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //--closescanner
    logformattitle = "调用CloseScanner方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    err = client.CloseScanner(scanresultnum)
    if err != nil {
        fmt.Printf("CloseScanner err:%s\n", err)
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //-------------------GetScannerResults

    logformattitle = "调用GetScannerResults方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis() //
    gsr, err := client.GetScannerResults([]byte(temptable), &hbase.TScan{
        StartRow: startrow,
        StopRow:  stoprow,
        //     FilterString: []byte("RowFilter(=, 'regexstring:00[1-3]00')"),
        //     FilterString: []byte("PrefixFilter('1407658495588-')"),
        Columns: []*hbase.TColumn{
            &hbase.TColumn{
                Family:    []byte("name"),
                Qualifier: []byte("idoall.org")}}}, 100)
    if err != nil {
        fmt.Printf("GetScannerResults err:%s\n", err)
    } else {
        fmt.Printf("GetScannerResults %d done\n", len(gsr))
        for _, k := range gsr {
            fmt.Println("scan Rowkey:" + string(k.Row))
            for _, cv := range k.ColumnValues {
                printscruct(cv)
            }
        }
    }

    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //---------------DeleteMultiple--------------
    logformattitle = "调用DeleteMultiple方法删除" + strconv.Itoa(TESTRECORD) + "数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    var tdelArr []*hbase.TDelete
    for i := 0; i < TESTRECORD; i++ {
        putrowkey := strconv.Itoa(i)
        tdelArr = append(tdelArr, &hbase.TDelete{
            Row: []byte(putrowkey)})
    }
    r, err := client.DeleteMultiple([]byte(temptable), tdelArr)
    if err != nil {
        fmt.Printf("DeleteMultiple err:%s\n", err)
    } else {
        fmt.Printf("DeleteMultiple %d done\n", TESTRECORD)
        fmt.Println(r)
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    endTime := currentTimeMillis()
    fmt.Printf("\nGolang调用总计用时:%d-%d=%d毫秒\n", endTime, startTime, (endTime - startTime))
}

func printscruct(cv interface{}) {
    switch reflect.ValueOf(cv).Interface().(type) {
    case *hbase.TColumnValue:
        s := reflect.ValueOf(cv).Elem()
        typeOfT := s.Type()
        //获取Thrift2中struct的field
        for i := 0; i < s.NumField(); i++ {
            f := s.Field(i)
            fileldformatstr := "\t%d: %s(%s)= %v\n"
            switch f.Interface().(type) {
            case []uint8:
                fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.Type(), string(f.Interface().([]uint8)))
            case *int64:
                var tempint64 int64
                if f.Interface().(*int64) == nil {
                    tempint64 = 0
                } else {
                    tempint64 = *f.Interface().(*int64)
                }
                fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.Type(), tempint64)
            default:
                fmt.Printf("I don't know")
            }
        }
    default:
        fmt.Printf("I don't know")
        fmt.Print(reflect.ValueOf(cv))
    }

}

func currentTimeMillis() int64 {
    return time.Now().UnixNano() / 1000000
}

  在hbase创建一个测试表“test_idoall_org”,用于使用Golang基于Thrift2的客户端来测试,因为Thrift2并没有提供创建和删除表的方法

root@m1:/home/hadoop# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase shell
2014-07-27 09:31:07,601 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):001:0> list
TABLE                                                                                                                                                                                                                  
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
0 row(s) in 2.8030 seconds
=> []
hbase(main):002:0> version
0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):003:0> status
2 servers, 0 dead, 1.0000 average load
hbase(main):004:0> create 'test_idoall_org','uid','name'
0 row(s) in 0.5800 seconds
=> Hbase::Table - test_idoall_org
hbase(main):005:0> list
TABLE                                                                                                                                                                                                                  
test_idoall_org                                                                                                                                                                                                        
1 row(s) in 0.0320 seconds
=> ["test_idoall_org"]
hbase(main):006:0> put 'test_idoall_org','10086','name:idoall','idoallvalue'
0 row(s) in 0.1090 seconds                 ^
hbase(main):009:0> get 'test_idoall_org','10086'
COLUMN                                                 CELL                                                                                                                                                            
 name:idoall                                           timestamp=1406424831473, value=idoallvalue                                                                                                                      
1 row(s) in 0.0450 seconds
hbase(main):010:0> scan 'test_idoall_org'
ROW                                                    COLUMN+CELL                                                                                                                                                     
 10086                                                 column=name:idoall, timestamp=1406424831473, value=idoallvalue                                                                                                  
1 row(s) in 0.0620 seconds
hbase(main):011:0>

  启动Hbase的Thrift2

root@m1:/home/hadoop/thrift_hbase# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift2
#在这里可以看到thrift2的日志记录到下面的路径中
starting thrift2, logging to /home/hadoop/hbase-0.96.2-hadoop2/bin/../logs/hbase-root-thrift2-m1.out
root@m1:/home/hadoop/thrift_hbase# jps
5705 Main
3614 ResourceManager
9426 HMaster
9944 Jps
3844 DFSZKFailoverController
9679 ThriftServer
2773 QuorumPeerMain
2944 JournalNode
root@m1:/home/hadoop/thrift_hbase#

  运行基于Thrift2的Golang客户端,在之前要先确保hbase已经打开,在浏览器输入:http://m1:60010/master-status,可以看到下图效果,如果看不到,去调试下hbase吧,还没有启动~~~
  
  运行hbase.go,如果你看到和我一样的提示信息,再次恭喜你我们的Golang也可以正常访问到Hbase了。

root@m1:/home/hadoop/thrift_hbase# go run hbase.go 
----建立连接 用时:1408639700869-1408639700867=2毫秒

----调用Exists方法
rowkey{1} in table{test_idoall_org} Exists:false
----调用Exists方法 用时:1408639700894-1408639700869=25毫秒

----调用Put方法写数据
Put done
----调用Put方法写数据 用时:1408639700905-1408639700894=11毫秒

----调用Get方法获取新增加的数据
Rowkey:1
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= welcome idoall.org
        3: Timestamp(*int64)= 1408639700843
----调用Get方法获取新增加的数据 用时:1408639700909-1408639700905=4毫秒

----调用Put update方法'修改'数据
Put update done
----调用Put update方法'修改'数据 用时:1408639700916-1408639700909=7毫秒

----调用Get方法获取'修改'后的数据
update Rowkey:1
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= welcome idoall.org---update
        3: Timestamp(*int64)= 1408639700858
----调用Get方法获取'修改'后的数据 用时:1408639700918-1408639700916=2毫秒

----调用DeleteSingle方法删除一条数据
DeleteSingel done
----调用DeleteSingle方法删除一条数据 用时:1408639700931-1408639700918=13毫秒

----调用PutMultiple方法添加10条数据
PutMultiple done
----调用PutMultiple方法添加10条数据 用时:1408639700941-1408639700931=10毫秒

----调用GetMultiple方法获取10数据
GetMultiple Count:10
Rowkey:0
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931138625 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:1
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931689108 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:2
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931691688 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:3
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931693392 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:4
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931694819 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:5
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931696588 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:6
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931698745 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:7
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931700279 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:8
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931702153 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:9
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931704203 +0800 CST
        3: Timestamp(*int64)= 1408639700881
----调用GetMultiple方法获取10数据 用时:1408639700950-1408639700941=9毫秒

----调用OpenScanner方法
OpenScanner 23 done
GetScannerRows 0 done
----调用OpenScanner方法 用时:1408639700956-1408639700950=6毫秒

----调用CloseScanner方法
----调用CloseScanner方法 用时:1408639700957-1408639700956=1毫秒

----调用GetScannerResults方法
GetScannerResults 0 done
----调用GetScannerResults方法 用时:1408639700961-1408639700958=3毫秒

----调用DeleteMultiple方法删除10数据
DeleteMultiple 10 done
[]
----调用DeleteMultiple方法删除10数据 用时:1408639700967-1408639700961=6毫秒


root@m1:/home/hadoop/thrift_hbase#

4. 常见问题

4.1 并发访问线程数

  首先,为了尽可能减少由于网络传输带来的时间开销,HBase的Thrift Server最好和应用客户端部署在同一台机器上。Thrift Server启动时可以通过参数配置并发线程数,否则很容易导致Thrift Server线程满了不响应客户端的读写请求,具体命令:/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase- daemon.sh start thrift –threadpool -m 200 -w 500(更多参数参考这里:/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift -h)。

4.2 最大堆内存配置

  如果客户端与Thrift Server进行scan操作顺序读取数据,而且设置了一定的cache记录条数(通过TScan的int32_t caching变量设置),那么这些被caching的记录数可能会占用Thrift Server相当部分的堆内存,尤其在多客户端并发访问时更明显。
  因此,在Thrift Server启动前,可以调大最大堆内存,否则可 能由于java.lang.OutOfMemoryError异常而导致进程被杀掉,尤其是当Scan时设置了较大的caching记录条数的情况(默认为expor t HBASE_HEAPSIZE=1000MB,可以在/home/hadoop/hbase-0.96.2-hadoop2/hbase- env.sh中设置)。

5. 扩展阅读

Thrift介绍与应用(三)—hbase的thrift接口


博文作者:迦壹
博客地址:PHP和Golang使 用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)
写作时间:2014-08-22 01:31
转载声明:可以转载, 但必须以超链接形式标 明文章原始出处和作者信息及版权声明,谢谢合作!


说点什么

您将是第一位评论人!

提醒
avatar
wpDiscuz