# spark2021-0829

**Repository Path**: wolfcub.com/spark2021-0829

## Basic Information

- **Project Name**: spark2021-0829
- **Description**: spark2021-0829
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-09-05
- **Last Updated**: 2021-09-05

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# spark2021-0829

#### 第一题-添加一个自定义命令
![作业1](https://images.gitee.com/uploads/images/2021/0905/222057_d53789d5_330140.png "WX20210905-175330.png")
#### 第二题-构建SQL满足如下要求
通过set spark.sql.planChangeLog.level=WARN;查看
1. 构建一条SQL，同时apply下面三条优化规则：
   CombineFilters-过滤条件组合
   CollapseProject-列合并
   BooleanSimplification-布尔表达式优化
2. 构建一条SQL，同时apply下面五条优化规则：
   ConstantFolding-常量折叠
   PushDownPredicates
   ReplaceDistinctWithAggregate
   ReplaceExceptWithAntiJoin
   FoldablePropagation

##### 第一个小题
> SELECT c.customerId FROM (SELECT customerId FROM sales WHERE 1=1 ) as c WHERE c.customerId>100 and c.customerId=1000
#### 第二小题
> SELECT DISTINCT c.customerId, 2 x  FROM (SELECT customerId FROM sales WHERE  amount > 5 + 3 AND 'a' = 'a' ) as c WHERE c.customerId>100 and c.customerId=1000
> EXCEPT SELECT customerId FROM sales ORDER BY x

#### 第三题 
[代码地址](https://gitee.com/wolfcub.com/spark2021-0829/blob/master/pushDown/src/main/scala/MySparkSessionExtension.scala)