Hadoop安全

Hadoop原本来自于谷歌一款名为MapReduce的编程模型包。谷歌的MapReduce框架可以把一个应用程序分解为许多并行计算指令，跨大量的计算节点运行很好巨大的数据集。使用该框架的一个典型例子就是在网络数据上运行的搜索算法。Hadoop很初只与网页索引有关，迅速发展成为分析大数据的靠前平台。目前有很多公司开始提供基于Hadoop的商业软件、支持、服务以及培训。本书讲述Hadoop相关安全技术。

本·斯皮维，Cloudera解决方案架构师，为客户提供Hadoop安全防护相关咨询服务。他与多个行业的财富500强公司合作过，包括金融、零售和健康服务行业。
乔伊·爱彻利维亚，Rocana公司软件工程师，在Hadoop平台上构建了IT运行分析系统。作为KiteSDK的提交者，他为多个项目贡献了代码，包括Apache Flume、Sqoop、Hadoop和HBase。

Foreword
Preface
1. Introduction
Security Overview
Confidentiality
Integrity
Availability
Authentication, Authorization, and Accounting
Hadoop Security: A Brief History
Hadoop Components and Ecosystem
Apache HDFS
Apache YARN
Apache MapReduce
Apache Hive
Cloudera Impala
Apache Sentry (Incubating)
Apache HBase
Apache Accumulo
Apache Solr
Apache Oozie
Apache ZooKeeper
Apache Flume
Apache Sqoop
Cloudera Hue
Summary
Part I.Security Architecture
2. Securing Distributed Systems
Threat Categories
Unauthorized Access/Masquerade
Insider Threat
Denial of Service
Threats to Data
Threat and Risk Assessment
User Assessment
Environment Assessment
Vulnerabilities
Defense in Depth
Summary
3. System Architecture
Operating Environment
Network Security
Network Segmentation
Network Firewalls
Intrusion Detection and Prevention
Hadoop Roles and Separation Strategies
Master Nodes
Worker Nodes
Management Nodes
Edge Nodes
Operating System Security
Remote Access Controls
Host Firewalls
SELinux
Summary
4. Kerberos
Why Kerberos?
Kerberos Overview
Kerberos Workflow: A Simple Example
Kerberos Trusts
MIT Kerberos
Server Configuration
Client Configuration
Summary
Part II.Authentication, Authorization, and Accounting
5. Identity and Authentication
Identity
Mapping Kerberos Principals to Usernames
Hadoop User to Group Mapping
Provisioning of Hadoop Users
Authentication
Kerberos
Username and Password Authentication
Tokens
Impersonation
Configuration
Summary
6. Authorization
HDFS Authorization
HDFS Extended ACLs
Service-Level Authorization
MapReduce and YARN Authorization
MapReduce (MR1)
YARN (MR2)
ZooKeeper ACLs
Oozie Authorization
HBase and Accumulo Authorization
System, Namespace, and Table-Level Authorization
Column- and Cell-Level Authorization
Summary
7. Apache Sentry (Incubating)
Sentry Concepts
The Sentry Service
Sentry Service Configuration
Hive Authorization
Hive Sentry Configuration
Impala Authorization
Impala Sentry Configuration
Solr Authorization
Solr Sentry Configuration
Sentry Privilege Models
SQL Privilege Model
Solr Privilege Model
Sentry Policy Administration
SQL Commands
SQL Policy File
Solr Policy File
Policy File Verification and Validation
Migrating From Policy Files
summary
8. Accounting
HDFS Audit Logs
MapReduce Audit Logs
YARN Audit Logs
Hive Audit Logs
Cloudera Impala Audit Logs
HBase Audit Logs
Accumulo Audit Logs
Sentry Audit Logs
Log Aggregation
Summary
Part III.Data Security
9. Data Protection
Encryption Algorithms
Encrypting Data at Rest
Encryption and Key Management
HDFS Data-at-Rest Encryption
MapReduce2 Intermediate Data Encryption
Impala Disk Spill Encryption
Full Disk Encryption
Filesystem Encryption
Important Data Security Consideration for Hadoop
Encrypting Data in Transit
Transport Layer Security
Hadoop Data-in-Transit Encryption
Data Destruction and Deletion
Summary
10. Securing Data Ingest
Integrity of Ingested Data
Data Ingest Confidentiality
Flume Encryption
Sqoop Encryption
Ingest Workflows
Enterprise Architecture
summary
11. Data Extraction and Client Access Security.
Hadoop Command-Line Interface
Securing Applications
HBase
HBase Shell
HBase REST Gateway
HBase Thrift Gateway
Accumulo
Accumulo Shell
Accumulo Proxy Server
Oozie
Sqoop
SQL Access
Impala
Hive
WebHDFS/HttpFS
Summary
12. Cloudera Hue
Hue HTTPS
Hue Authentication
SPNEGO Backend
SAML Backend
LDAP Backend
Hue Authorization
Hue SSL Client Configurations
Summary
Part IV.Putting It All Together
13. Case Studies
Case Study: Hadoop Data Warehouse
Environment Setup
User Experience
Summary
Case Study: Interactive HBase Web Application
Design and Architecture
Security Requirements
Cluster Configuration
Implementation Notes
Summary
Afterword
Index