Ucloud UDDP(Hadoop + hive)测试小记

尘缘Hadoop

数据结构：150-200字节/行，平均160

+-----------------------+-----------------------+-----------------------+
|       col_name        |       data_type       |        comment        |
+-----------------------+-----------------------+-----------------------+
| ip                    | string                | None                  |
| time                  | int                   | None                  |
| day                   | int                   | None                  |
| act                   | string                | None                  |
| channel               | string                | None                  |
| vcode                 | int                   | None                  |
| imei                  | string                | None                  |
| mc                    | string                | None                  |
| md5                   | string                | None                  |
| android_id            | string                | None                  |
| model                 | string                | None                  |
| refer                 | string                | None                  |
| push                  | int                   | None                  |
+-----------------------+-----------------------+-----------------------+

文件清单：

+----------------------------------------+
|               DFS Output               |
+----------------------------------------+
| 15147471000   /umr-mpnp2o/log          |
| 45442413000   /umr-mpnp2o/log02        |
| 454424130000  /umr-mpnp2o/log03        |
| 0             /umr-mpnp2o/uhivebh5eoz  |
+----------------------------------------+

Uhost to UDDP 文件写入速度：

+-----------------------+-----------------------+-----------------------+
|  100000000 rows       |    300000000 rows     |    3000000000 rows    |
+-----------------------+-----------------------+-----------------------+
| 14G                   | 42G                   | 423G                  |
| put time 6m32.212s    | put time 18m42.742s   | put time 187m11.383s  |
| count time 32.501s    | count time 46.815s    | count time 204.257s   |
+-----------------------+-----------------------+-----------------------+