<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0">
<channel>
<title><![CDATA[向东博客 专注WEB应用 构架之美 --- 构架之美，在于尽态极妍 | 应用之美，在于药到病除]]></title> 
<link>http://www.jackxiang.com/index.php</link> 
<description><![CDATA[赢在IT，Playin' with IT,Focus on Killer Application,Marketing Meets Technology.]]></description> 
<language>zh-cn</language> 
<copyright><![CDATA[向东博客 专注WEB应用 构架之美 --- 构架之美，在于尽态极妍 | 应用之美，在于药到病除]]></copyright>
<item>
<link>http://www.jackxiang.com/post//</link>
<title><![CDATA[[实践OK]Linux出现Too many open files in system强制重启，原因是恶意用户连接SSH导致ssh-agent进程太多，最后只有reset键和长按关机键关机。]]></title> 
<author>jack &lt;xdy108@126.com&gt;</author>
<category><![CDATA[Unix/LinuxC技术]]></category>
<pubDate>Wed, 16 May 2018 14:51:12 +0000</pubDate> 
<guid>http://www.jackxiang.com/post//</guid> 
<description>
<![CDATA[ 
	原因：外网试图登录SSH破解密码太多，出现大量的/usr/bin/ssh-agent -s进程。ssh-agent会随着当前ssh会话的消失而消失，这也是一种安全机制.<br/>解决办法如下：<br/>cat&nbsp;&nbsp;/etc/profile.d/ssh-agent.sh&nbsp;&nbsp; <br/><textarea name="code" class="php" rows="15" cols="100"> 
#!/bin/sh
if [ -f ~/.agent.env ]; then
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;. ~/.agent.env &gt;/dev/null
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if ! kill -0 $SSH_AGENT_PID &gt;/dev/null 2&gt;&amp;1; then
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;echo &quot;Stale agent file found. Spawning new agent...&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;eval `ssh-agent &#124;tee ~/.agent.env`
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ssh-add
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fi
else
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;echo &quot;Starting ssh-agent...&quot;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;eval `ssh-agent &#124;tee ~/.agent.env`
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ssh-add
fi
</textarea><br/>运行：<br/>[root@localhost ~]# sh&nbsp;&nbsp;/etc/profile.d/ssh-agent.sh&nbsp;&nbsp; <br/>Starting ssh-agent...<br/>Agent pid 3625<br/>[root@localhost ~]# ps -ef&#124;grep 3625<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3625&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:57 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 ssh-agent<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3647&nbsp;&nbsp;3325&nbsp;&nbsp;0 22:57 pts/2&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 grep --color=auto 3625<br/><br/>后面再多个SSH，只有四个ssh-agent进程:<br/>ps -ef&#124;grep ssh-agent&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3625&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:57 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 ssh-agent<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2442&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:54 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 /usr/bin/ssh-agent -s<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2785&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:55 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 /usr/bin/ssh-agent -s<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3131&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:56 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 /usr/bin/ssh-agent -s<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3527&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:57 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 /usr/bin/ssh-agent -s<br/>root&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3878&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;0 22:58 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;00:00:00 /usr/bin/ssh-agent -s<br/><br/>cat ~/.agent.env<br/>SSH_AUTH_SOCK=/tmp/ssh-X1RtYLOVI2iq/agent.3623; export SSH_AUTH_SOCK;<br/>SSH_AGENT_PID=3625; export SSH_AGENT_PID;<br/>echo Agent pid 3625;<br/><br/>感觉进程数还是变多，无鸟用，于是干掉这个SSH-Client：<br/>[root@localhost ~]# rpm -qf /usr/bin/ssh-agent<br/>openssh-clients-7.4p1-13.el7_4.x86_64<br/>[root@localhost ~]# rpm -e openssh-clients<br/>错误：依赖检测失败：<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;openssh-clients 被 (已安裝) virt-viewer-5.0-7.el7.x86_64 需要<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;openssh-clients 被 (已安裝) python-meh-0.25.2-1.el7.noarch 需要<br/>[root@localhost ~]# yum remove&nbsp;&nbsp;openssh-clients -y<br/>作为依赖被删除:<br/>&nbsp;&nbsp;anaconda-core.x86_64 0:21.48.22.121-1.el7.centos&nbsp;&nbsp;&nbsp;&nbsp;anaconda-gui.x86_64 0:21.48.22.121-1.el7.centos&nbsp;&nbsp; <br/>&nbsp;&nbsp;anaconda-tui.x86_64 0:21.48.22.121-1.el7.centos&nbsp;&nbsp;&nbsp;&nbsp; initial-setup.x86_64 0:0.3.9.40-1.el7.centos&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br/>&nbsp;&nbsp;initial-setup-gui.x86_64 0:0.3.9.40-1.el7.centos&nbsp;&nbsp;&nbsp;&nbsp;python-meh.noarch 0:0.25.2-1.el7&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br/>&nbsp;&nbsp;python-meh-gui.noarch 0:0.25.2-1.el7&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;virt-viewer.x86_64 0:5.0-7.el7 <br/><br/>参考自：https://blog.csdn.net/diamondxiao/article/details/52488628<br/><br/>一）SSH现象：<br/><textarea name="code" class="php" rows="15" cols="100">
ps -ef&#124;grep ssh-agent&#124;wc -l
-bash: 管道错误: Too many open files in system
-bash: start_pipeline: 进程组管道: Too many open files in system
lsof -n &#124;awk &#039;&#123;print $2&#125;&#039;&#124;sort&#124;uniq -c &#124;sort -nr&#124;more&nbsp;&nbsp; 
-bash: /usr/bin/sort: Too many open files in system
-bash: /usr/bin/sort: Too many open files in system
lsof: error while loading shared libraries: libdl.so.2: cannot open shared object file: Error 23
rpm -qa&#124;grep ssh-agent
error: Failed to initialize NSS library
[xiangdong@localhost ~]$ yum update nspr
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
libz.so.1: cannot open shared object file: Too many open files in system
reboot 
/usr/bin/pkttyagent: error while loading shared libraries: libffi.so.6: cannot open shared object file: Error 23
Failed to execute operation: 连接超时
sudo su -
上一次登录：一 5月 14 23:12:47 CST 2018:0 上
最后一次失败的登录：三 5月 16 22:27:19 CST 2018从 218.65.30.53ssh:notty 上
最有一次成功登录后有 64523 次失败的登录尝试。
[root@localhost ~]# reboot 
uptime
-bash: start_pipeline: 进程组管道: Too many open files in system
-bash: /usr/bin/uptime: Too many open files in system
id
-bash: start_pipeline: 进程组管道: Too many open files in system
-bash: /usr/bin/id: Too many open files in system
ps -ef&#124;grep ssh
-bash: 管道错误: Too many open files in system
-bash: start_pipeline: 进程组管道: Too many open files in system
</textarea><br/><br/><br/>二）Linux桌面没了，出现：<br/>系统出现kernel: audit: backlog limit exceeded提示<br/>error: audit:backlog limit exceeded报错，audit缓冲大小瓶颈<br/>来自：http://blog.51cto.com/jschu/1769025<br/><br/>三）如何强制重启？<br/>有时候,linux 由于硬盘或者其它原因，<br/>某个进程挂住了，怎么也杀不死，<br/>输入 reboot 命令也无法重启。<br/>这时候，一般只能按机箱上的reset健来重启了。<br/> 如果是远程ssh，那怎么办呢？<br/><br/><br/> 下面两行命令可以搞定：<br/><br/><br/>echo 1 &gt; /proc/sys/kernel/sysrq<br/>echo b &gt; /proc/sysrq-trigger<br/>=========================================================<br/>Linux使用shutdown -r now 或者 reboot、init 6 命令无法重启时使用以下两条命令可强制重启：<br/>echo 1 &gt; /proc/sys/kernel/sysrq&nbsp;&nbsp;<br/>echo b &gt; /proc/sysrq-trigger&nbsp;&nbsp;<br/>1./proc/sys/kernel/sysrq<br/>向 sysrq 文件中写入1是为了开启 SysRq 功能。根据 linux/Documentations/sysrq.txt 中所说：SysRq 代表的是 Magic System Request Key。开启了这个功能以后，只要内核没有挂掉，它就会响应你要求的任何操作。但是这需要内核支持(CONFIG_MAGIC_SYSRQ 选项)。向 /proc/sys/kernel/sysrq 中写入0是关闭 SysRq 功能，写入1是开启，其他选项请参考 sysrq.txt。<br/><br/>2./proc/sysrq-trigger<br/>立即重新启动计算机:<br/>echo &quot;b&quot; &gt; /proc/sysrq-trigger<br/>立即关闭计算机：<br/>echo &quot;o&quot; &gt; /proc/sysrq-trigger<br/><br/><br/><br/>来自：https://my.oschina.net/dongsong/blog/915358<br/><br/>
]]>
</description>
</item><item>
<link>http://www.jackxiang.com/post//#blogcomment</link>
<title><![CDATA[[评论] [实践OK]Linux出现Too many open files in system强制重启，原因是恶意用户连接SSH导致ssh-agent进程太多，最后只有reset键和长按关机键关机。]]></title> 
<author> &lt;user@domain.com&gt;</author>
<category><![CDATA[评论]]></category>
<pubDate>Thu, 01 Jan 1970 00:00:00 +0000</pubDate> 
<guid>http://www.jackxiang.com/post//#blogcomment</guid> 
<description>
<![CDATA[ 
	
]]>
</description>
</item>
</channel>
</rss>