User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; .NET4.0C) (This is the bug as submitted to the EGI SVG - see additional info.) Hello, I represent Polish NGI (security officer). I would like to report a vulnerability we've found in Torque. Below is my original email to Torque developers. This issue concerns PBS Pro (at least last version) as well. best regards - -- Bartlomiej Balcerek Wroclaw University of Technology, Poland Wroclaw Centre for Networking and Supercomputing phone: +48 (71) 320-20-79 mail: bartol.pl [lookup email] [lookup "pwr.wroc.pl"] - ------ Wiadomość oryginalna ------ Temat: Security vulnerability in Torque Data: Tue, 10 May 2011 10:11:19 +0200 Nadawca: Bartlomiej Balcerek <bartol.pl> [lookup email] [lookup "pwr.wroc.pl"] Firma/Organizacja: WCSS, PWr. Adresat: info [lookup email] [lookup "clusterresources.com"] Kopia: maciej.kotowicz.pl [lookup email] [lookup "pwr.wroc.pl"], adam.zabrocki.pl [lookup email] [lookup "pwr.wroc.pl"] Dear Sirs, I represent Wroclaw Centre for Networking and Supercomputing Security Team. Recently, during the security audit, we discovered a bug in Torque server and pbs_iff (all versions) which can have serious implications to security. This is likely to lead attacker to elevate his privileges. Torque server does not check the length of "job name" argument before using it - this string is verified only on the client side. It is possible to use modified Torque client or DRMAA interface to submit job with arbitrary chosen job name in terms of length and content. Thus, it is possible to attacker to overflow buffer and overwrite some Torque server process internal data causing its specific behavior. Below are some technical details concerning x86_64 build: What can be overwritten is log_buffer global string array and all next symbols: 0000000000734b00 B log_buffer 0000000000738b00 B msg_registerrel 0000000000738b08 B msg_manager 0000000000738b10 B msg_startup1 0000000000738b18 B msg_momnoexec1 0000000000738b20 B msg_man_uns 0000000000738b28 B msg_sched_nocall 0000000000738b30 B msg_issuebad 0000000000738b38 B stdout@@GLIBC_2.2.5 0000000000738b40 B msg_job_end_stat 0000000000738b48 b dtor_idx.6147 0000000000738b50 b completed.6145 0000000000738b58 b acct_opened 0000000000738b5c b acct_auto_switch 0000000000738b60 b acctfile 0000000000738b68 b acct_opened_day 0000000000738b70 b spaceused 0000000000738b78 b spaceavail 0000000000738b80 b username.6360 0000000000738bc0 b groupname.6402 Here I submit the crafted job: [bartol@bartek_torque torque-mod]$ echo /bin/date | ./src/cmds/qsub -Z "Job_Name=`perl -e 'print "A"x16350'`" It is possible now to see in debugger that structures adjacent to log_buffer are overwritten with "A" chars (encoded as 0x41 numbers): Program received signal SIGINT, Interrupt. 0x00000033550cd323 in __select_nocancel () from /lib64/libc.so.6 (gdb) x/20x 0x0000000000738b00 0x738b00 <msg_registerrel>: 0x4141414141414141 0x4141414141414141 0x738b10 <msg_startup1>: 0x4141414141414141 0x4141414141414141 0x738b20 <msg_man_uns>: 0x4141414141414141 0x4141414141414141 The overflow occurs in the following code: 1560 sprintf(log_buffer, msg_jobnew, 1561 preq->rq_user, preq->rq_host, 1562 pj->ji_wattr[(int)JOB_ATR_job_owner].at_val.at_str, 1563 pj->ji_wattr[(int)JOB_ATR_jobname].at_val.at_str, 1564 pj->ji_qhdr->qu_qs.qu_name); We proved that server crash is easily possible (including database damage) and we think privilege escalation can be done with some more effort as well, but the latter is strongly dependable on particular build flags and architecture. The overflow is also possible in pbs_iff setuid binary, since the "host" variable length is not checked: sprintf(log_buffer,"cannot resolve IP address for host '%s' herror=%d: %s", hostname, /*1*/ h_errno, hstrerror(h_errno)); Please let us know if you need some additional information and what are your plans concerning this issue. Bartlomiej Balcerek Maciej Kotowicz Adam Zabrocki Reproducible: Always This problem was reported to the EGI Software Vulnerability Group by Bartlomiej.Balcerek (Bartlomiej.Balcerek.pl) rather than found by the submitter. It has also been reported to the Torque providers, and been fixed by them - an e-mail from Douglas Wightman (wightman) states: Dear Bartlomiej, Thank you for the email. I apologize that we did not reply sooner. Our Torque engineers have patched this vulnerability in the latest releases of Torque (2.4 and higher). We are working on getting you a new release that you can verify. Please expect a follow-up email soon. Thanks for your help with this issue and please continue to report all future issues that you may encounter. Thanks, - Douglas Wightman This bug is being entered in order that a 'fixed' version of Torque is available in EPEL. The EGI http://www.egi.eu/ Software Vulnerability group http://www.egi.eu/policy/groups/Software_Vulnerability_Group_SVG.html runs a process for handling software vulnerabilities reported. While our work is primarily designed to handle vulnerabilities in Grid Middleware, other vulnerabilities found in software used in the EGI infrastructure may also be reported to us and we pass the information on to the software suppliers, as well as considering the risk to the EGI infrastructure.
Thanks for the report, I should have new packages in the next day or so for EPEL 4, EPEL5, EPEL6 , Fedora 14, Fedora 15 and Fedora 16 where this vulnerability applies. Looking upstream I can see a fix submitted to HEAD for this but it does not look to be released AFAICT in any of their releases as yet. I've back-ported their patch back to the EPEL version as well. Will push updates to testing in a day or so at same time as making this bug public.
Created attachment 503747 [details] Trivial patch for buffer overrun This is patch pulled from trunk on SVN. svn diff -r 4680:4681 svn://svn.clusterresources.com/torque/trunk
The upstream fix can be found via: svn diff -r 4705:4706 svn://clusterresources.com/torque/ I'm assigning this CVE-2011-2193. If someone could tell upstream, that would be handy. I would like to open this bug up now. Since the fix is public (and clearly marked as fixing a buffer overflow in svn). Thanks
Yes: 4705:4706 is correct, I was using another branch but it's the correct fix I had to redo the patch anyway for everything older than Fedora 15. Pushing packages now.
Hi, Please open the bug up, it seems all my bodhi updates failed to update this bug with: Update successfully created. Unable to access one or more bugs: You are not authorized to access bug #711463.. :-) Anyway updates are here: https://admin.fedoraproject.org/updates/torque-2.3.13-2.el4 https://admin.fedoraproject.org/updates/torque-2.3.13-2.el5 https://admin.fedoraproject.org/updates/torque-2.5.5-2.el6 https://admin.fedoraproject.org/updates/torque-2.4.11-2.fc14 https://admin.fedoraproject.org/updates/torque-3.0.1-2.fc15
It should be public now.
torque-2.3.13-2.el4 has been submitted as an update for Fedora EPEL 4. https://admin.fedoraproject.org/updates/torque-2.3.13-2.el4
torque-2.3.13-2.el5 has been submitted as an update for Fedora EPEL 5. https://admin.fedoraproject.org/updates/torque-2.3.13-2.el5
torque-2.5.5-2.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/torque-2.5.5-2.el6
torque-2.4.11-2.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/torque-2.4.11-2.fc14
torque-3.0.1-2.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/torque-3.0.1-2.fc15
Package torque-2.5.5-2.el6: * should fix your issue, * was pushed to the Fedora EPEL 6 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=epel-testing torque-2.5.5-2.el6' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/torque-2.5.5-2.el6 then log in and leave karma (feedback).
torque-2.4.11-2.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report.
torque-2.3.13-2.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report.
torque-2.3.13-2.el4 has been pushed to the Fedora EPEL 4 stable repository. If problems still persist, please make note of it in this bug report.
torque-2.5.5-2.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.
torque-3.0.1-4.fc15 has been pushed to the Fedora 15 stable repository. If problems still persist, please make note of it in this bug report.