高高兴兴地来上班，突闻任务大面积报错，经过一番调查，发现服务器上的某个用户不见了。 
这是可坏的，某azkaban这个用户执行上面的所有任务，这个用户下面有很多crontab任务，全部都执行不了，包括给大boss发的邮件。
我的第一反应是快速添加用户，领导说等ldap自动同步，检查ldap服务，发现没问题，但连不上ldap服务器，网络丢包严重，等网络组处理。~ 在此期间，由于网络问题，我使用其他用户执行脚本。 等啊等，等啊等… 运维哥等不及了，手动添加了这个用户，但脚本还是执行不了。报告如下错误: Exception in thread “main” java.lang.RuntimeException: java.io.IOException: Permission denied Caused by: java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createTempFile(File.java:2024) 查了下hive参数hive.exec.scratchdir设置目录:/tmp/hive 权限没有问题 然后查看hive日志/tmp/xx用户/hive.log，发现要创建本地目录 本地目录的用户和用户组都是数字，数字被删除uid和gid，虽然新用户的名称与被删除用户相同，但id不一样。 当然，上述错误不一定是同样的原因，具体原因取决于日志。
至于用户为什么消失，运维给出的结论是网络异常，ldap服务器无法访问。我不太明白这一点。即使网络不通，用户也不会被删除，对吧？有可能被删除，无法同步。 Anyway，只要有人背锅，人为是否不那么重要。
PS: 我测试过，只要用户被删除，原本属于这个用户的文件就会变成id 被删前?? 被删后?? 详细日志： INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/xx/28651db7-6f1b-4162-9ea4-cafe969c335b INFO [main] session.SessionState: Created local directory: /hive/local/xx/28651db7-6f1b-4162-9ea4-cafe969c335b INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/xx/28651db7-6f1b-4162-9ea4-cafe969c335b/_tmp_space.db INFO [main] conf.HiveConf: Using the default value passed in for log id: 28651db7-6f1b-4162-9ea4-cafe969c335b INFO [main] session.SessionState: Updating thread name to 28651db7-6f1b-4162-9ea4-cafe969c335b main INFO [28651db7-6f1b-4162-9ea4-cafe969c335b main] CliDriver: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
2022-04-20T21:26:21,302 INFO [21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 main] CliDriver: Time taken: 4.602 seconds, Fetched: 1 row(s) 2022-04-20T21:26:21,302 INFO [21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 main] conf.HiveConf: Using the default value passed in for log id: 21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 2022-04-20T21:26:21,302 INFO [21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 main] session.SessionState: Resetting thread name to main 2022-04-20T21:26:21,302 INFO [main] conf.HiveConf: Using the default value passed in for log id: 21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 2022-04-20T21:26:21,306 INFO [main] session.SessionState: Deleted directory: /tmp/hive/xx/21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 on fs with scheme hdfs 2022-04-20T21:26:21,311 INFO [main] session.SessionState: Deleted directory: /hive/local/xx/21ec89d4-93b5-44e3-a8ba-0fc7ecfbbb46 on fs with scheme file 2022-04-20T21:26:21,312 INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore… 2022-04-20T21:26:21,312 INFO [main] HiveMetaStore.audit: ugi=xx ip=unknown-ip-addr cmd=Cleaning up thread local RawStore… 2022-04-20T21:26:21,312 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2022-04-20T21:26:21,312 INFO [main] HiveMetaStore.audit: ugi=xx ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2022-04-21T00:00:14,722 INFO [main] metadata.Hive: Registering function row_seq com.bigdata.hive.udf.impl.HLSequenceGenerator 2022-04-21T00:00:14,723 WARN [main] metadata.Hive: Failed to register persistent function row_seq:com.bigdata.hive.udf.impl.HLSequenceGenerator. Ignore and continue. 2022-04-21T00:00:14,723 INFO [main] metadata.Hive: Registering function decrypt_hyd com.credithc.udf.DecryptHydDataFunction 2022-04-21T00:00:14,724 INFO [main] metadata.Hive: Registering function udfsharesc org.yjy.udfshare.udfsharesc 2022-04-21T00:00:14,725 INFO [main] metadata.Hive: Registering function fieldcrc32 com.cn.HiveUDF 2022-04-21T00:00:14,726 WARN [main] metadata.Hive: Failed to register persistent function fieldcrc32:com.cn.HiveUDF. Ignore and continue. 2022-04-21T00:00:14,726 INFO [main] metadata.Hive: Registering function decrypt_mobile com.credithc.udf.DecryptBdpDataFunction 2022-04-21T00:00:14,726 INFO [main] metadata.Hive: Registering function decrypt_mobile com.credithc.udf.DecryptBdpDataFunction 2022-04-21T00:00:14,727 INFO [main] metadata.Hive: Registering function parse_json com.hc.udf.JsonPar 2022-04-21T00:00:14,727 WARN [main] metadata.Hive: Failed to register persistent function parse_json:com.hc.udf.JsonPar. Ignore and continue. 202204-21T00:00:14,727 INFO [main] metadata.Hive: Registering function parse_json com.hc.udf.JsonPar 2022-04-21T00:00:14,728 WARN [main] metadata.Hive: Failed to register persistent function parse_json:com.hc.udf.JsonPar. Ignore and continue. 2022-04-21T00:00:14,728 INFO [main] metadata.Hive: Registering function getduration com.hc.HUGetDuration 2022-04-21T00:00:14,734 INFO [main] metadata.Hive: Registering function encrypt_bdp com.credithc.udf.EncryptBdpDataFunction 2022-04-21T00:00:14,734 INFO [main] metadata.Hive: Registering function addfive com.atguigu.myudf.MyUdf 2022-04-21T00:00:14,735 WARN [main] metadata.Hive: Failed to register persistent function addfive:com.atguigu.myudf.MyUdf. Ignore and continue. 2022-04-21T00:00:14,735 INFO [main] metadata.Hive: Registering function splitstr com.atguigu.myudf.MyUdf 2022-04-21T00:00:14,736 WARN [main] metadata.Hive: Failed to register persistent function splitstr:com.atguigu.myudf.MyUdf. Ignore and continue. 2022-04-21T00:00:14,736 INFO [main] metadata.Hive: Registering function splitstr2 com.atguigu.myudtf.MyUdtf 2022-04-21T00:00:14,736 WARN [main] metadata.Hive: Failed to register persistent function splitstr2:com.atguigu.myudtf.MyUdtf. Ignore and continue.