Hive是一种建立在Hadoop文件系统上的数据仓库架构,并对存储在HDFS中的数据进行分析和管理;本文通过一个简单的示例来展现如何结合OSS+EMR+Hive来分] c M 3析OSS上的数据。
前提条件
• 已注册阿里云账号,详情请参见注册云账号。
• 已开通E-MapReduce服务和OSS服务。
• 已完成云账号的授权,详情请参见角色授权。
• 已创建Haoop集群,P d p z Z 0且带有Hive组件, 且配置好OSS数- G S i v据源。
步骤一:上传数据至OSS
hadoop fs -put course.csv oss://your-bucket-name/
步骤二:创建Hive作业开发页面
步t 4 + c骤三:SQL 实现
创建数据表
CREATE TABLE course (num INT, subject string, level string) row format delimited fields terminated by "," location "/";
导入数据
LOAD DATA INPATH 'oss://your-bucket-name/course.csv' INTOk ~ y * k T TABLE course;
查询语句
select A.suject, A.level, A.count from (select suject, level, count(*)
as count fru W . M K 3 Rom course group by suject , level) A join (selecV b : ^t B.suject as suject,
max(B.count) as cog $ O T z o }unt from (select suject, level, count(*) as count from course
group by suject , level) B group by B.suject ) C on A.suject=C.suject and A.count = C.count;; l s f n
发表评论