From d@scry.dstc.edu.au Sat Aug 31 00:02:39 1996 Received: from trapdoor.dstc.edu.au (root@trapdoor.dstc.edu.au [130.102.176.12]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id AAA26172 for ; Sat, 31 Aug 1996 00:02:37 -0700 (PDT) Received: from scry.dstc.edu.au (scry.dstc.edu.au [130.102.176.222]) by trapdoor.dstc.edu.au (8.6.9/8.6.12) with ESMTP id RAA13017 for ; Sat, 31 Aug 1996 17:02:34 +1000 Received: (from root@localhost) by scry.dstc.edu.au (8.7.5/8.6.12) id RAA15403; Sat, 31 Aug 1996 17:02:34 +1000 (EST) Message-Id: <199608310702.RAA15403@scry.dstc.edu.au> Date: Sat, 31 Aug 1996 17:02:34 +1000 (EST) From: David Leonard Reply-To: leonard@dstc.edu.au To: FreeBSD-gnats-submit@freebsd.org Subject: pkg_add's auto dependency get .. doesn't work X-Send-Pr-Version: 3.2 >Number: 1557 >Category: bin >Synopsis: pkg_add's auto dependency get .. doesn't work >Confidential: no >Severity: serious >Priority: low >Responsible: jkh >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Aug 31 00:10:01 PDT 1996 >Closed-Date: Mon Oct 14 12:22:08 PDT 1996 >Last-Modified: Mon Oct 14 12:41:19 PDT 1996 >Originator: David Leonard >Release: FreeBSD 2.2-CURRENT i386 >Organization: DSTC, Brisbane, Australia +61 7 3365 4310 >Environment: sup from sup.au.freebsd.org yesterday >Description: When you pkg_add something from a URL when the dependencies aren't all installed, pkg_add will attempt to auto-install the dependent packages for you. but it doesn't work. >How-To-Repeat: without having autoconf or m4 installed, i did # pkg_add -v "ftp://ftp/u9/freebsd/packages-2.1.5/All/autoconf-2.10.tgz" Trying to fetch ftp://ftp/u9/freebsd/packages-2.1.5/All/autoconf-2.10.tgz. Extracting from FTP connection into /var/tmp/instmp.014625 +CONTENTS [... and so on ...] share/autoconf/config.sub tar command returns 0 status Package `autoconf-2.10' depends on `m4-1.4'. Trying to fetch ftp://ftp/u9/freebsd/packages-2.1.5/All/m4-1.4. Error: FTP Unable to get ftp://ftp/u9/freebsd/packages-2.1.5/All/m4-1.4 Segmentation fault (core dumped) So, i looked into why the .tgz extension isn't being added in, and added that (see patches to pkg_install/lib/file.c below) then retried after a similar output it died on a 'Broken Pipe' signal after trying to fetch the right file. (i can't cut and paste because vi trashed the scrollback buffer) anyway, i traced this to a thing inside libftpio where with ftpGetURL, an static FILE* is being closed on each re-open. I suspect that the fclose done in ftpGetURL() is to avoid exhausting file descriptors... anyway what i have now works, even though some FILE*'s are left unclosed. i could not see straight away where the sigpipe was coming from... should have kept the core :( >Fix: ------------------------------------------------------------ for src/lib/libftpio/ Index: ftpio.c =================================================================== RCS file: /home/leonard/cvsroot/freebsd/src/lib/libftpio/ftpio.c,v retrieving revision 1.1.1.3 diff -c -r1.1.1.3 ftpio.c *** ftpio.c 1996/08/30 17:35:30 1.1.1.3 --- ftpio.c 1996/08/31 06:49:51 *************** *** 292,304 **** { char host[255], name[255]; int port; ! static FILE *fp = NULL; FILE *fp2; if (fp) { /* Close previous managed connection */ fclose(fp); fp = NULL; } if (get_url_info(url, host, &port, name) == SUCCESS) { fp = ftpLogin(host, user, passwd, port, 0); if (fp) { --- 292,307 ---- { char host[255], name[255]; int port; ! /* static */ FILE *fp = NULL; FILE *fp2; + #if 0 if (fp) { /* Close previous managed connection */ fclose(fp); fp = NULL; } + #endif + if (get_url_info(url, host, &port, name) == SUCCESS) { fp = ftpLogin(host, user, passwd, port, 0); if (fp) { ------------------------------------------------------------ for src/usr.sbin/pkg_install/lib might also want to add the '.tar.gz' case too? Index: file.c =================================================================== RCS file: /home/leonard/cvsroot/freebsd/src/usr.sbin/pkg_install/lib/file.c,v retrieving revision 1.1.1.1 diff -c -r1.1.1.1 file.c *** file.c 1996/08/16 00:20:37 1.1.1.1 --- file.c 1996/08/31 06:50:07 *************** *** 244,251 **** snprintf(pword, HOSTNAME_MAX + 40, "%s@%s", pw->pw_name, me); } if (Verbose) ! printf("Trying to fetch %s.\n", fname); ftp = ftpGetURL(fname, uname, pword); if (ftp) { pen[0] = '\0'; if ((rp = make_playpen(pen, 0)) != NULL) { --- 244,261 ---- snprintf(pword, HOSTNAME_MAX + 40, "%s@%s", pw->pw_name, me); } if (Verbose) ! printf("Trying to fetch %s\n", fname); ftp = ftpGetURL(fname, uname, pword); + + /* try adding a .tgz extension if there isn't one already */ + if (!ftp && 0!=strcmp( fname+(strlen(fname)-4), ".tgz") ) + { + strcat( fname, ".tgz" ); + if (Verbose) + printf("That failed, so let's try %s\n", fname); + ftp = ftpGetURL( fname, uname, pword ); + } + if (ftp) { pen[0] = '\0'; if ((rp = make_playpen(pen, 0)) != NULL) { >Release-Note: >Audit-Trail: From: "Jordan K. Hubbard" To: leonard@dstc.edu.au Cc: FreeBSD-gnats-submit@freebsd.org Subject: Re: bin/1557: pkg_add's auto dependency get .. doesn't work Date: Sat, 31 Aug 1996 14:23:51 -0700 > So, i looked into why the .tgz extension isn't being added in, and > added that (see patches to pkg_install/lib/file.c below) then > retried Genuine bug, thanks. > anyway, i traced this to a thing inside libftpio where with > ftpGetURL, an static FILE* is being closed on each re-open. > > I suspect that the fclose done in ftpGetURL() is to avoid > exhausting file descriptors... anyway what i have now works, Erm, sort of. :-( That construct is very deliberately the way it is, due to the fact that when you open a connection to an FTP, that returns one FILE*. If you then ask for a file from that connection, it returns another FILE*. Now, and this is the important part, if you close the first file pointer before finishing with the second, you lose. The intended use of ftpGetURL() was that you'd process the file it returned all the way to completion and close it before asking for another URL. This is clearly not what pkg_add is doing, and pkg_add has a bug I'll have to look into. Leaving leaks in libftpio is not an acceptable solution by any conceivable stretch. However, you have suggested an important optimization to me, and that's to cache the old host connection when getting ftpGetURL() requests back-to-back for the same host. That just makes simple sense, and was easy to do. I'll commit the changes shortly to libftpio, as well as some fixes to pkg_add. Jordan Responsible-Changed-From-To: freebsd-bugs->jkh Responsible-Changed-By: wosch Responsible-Changed-When: Thu Sep 26 17:08:00 PDT 1996 Responsible-Changed-Why: pkg_* is Jordans area. State-Changed-From-To: open->closed State-Changed-By: jkh State-Changed-When: Mon Oct 14 12:22:08 PDT 1996 State-Changed-Why: There were indeed entomological artifacts in this code - thanks for the bug report. Fixed! >Unformatted: