Debian Bug report logs - #681243
dpkg: warn when different packages have same file as directory and symlink

Reply or subscribe to this bug.

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Bart Martens <bartm@debian.org>

To: submit@bugs.debian.org

Subject: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 16:53:12 +0000

Package: dpkg
Version: 1.16.7

The file /usr/include/tcl is a directory or a symbolic link depending on the
order of installation of the packages tk-tile 0.8.2-2.2 and tcl-dev 8.5.0-2.

This can be reproduced quite easily with these commands:

  pbuilder --login
  apt-get -y install file 
  apt-get -y install tk-tile
  apt-get -y install tcl-dev
  file /usr/include/tcl
  apt-get -y --purge remove tk-tile
  apt-get -y --purge remove tcl-dev
  apt-get -y install tcl-dev
  apt-get -y install tk-tile
  file /usr/include/tcl
  dpkg -c /var/cache/apt/archives/tk-tile_0.8.2-2.2_amd64.deb | grep include/tcl
  dpkg -c /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb | grep include/tcl

In my opinion, regardless of the order in which these two packages are
installed, dpkg should fail to install the second package, with an error
message about /usr/include/tcl.

Regards,

Bart Martens

Message #10 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Sven Joachim <svenjoac@gmx.de>

To: Bart Martens <bartm@debian.org>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 19:07:49 +0200

On 2012-07-11 18:53 +0200, Bart Martens wrote:

> Package: dpkg
> Version: 1.16.7
>
> The file /usr/include/tcl is a directory or a symbolic link depending on the
> order of installation of the packages tk-tile 0.8.2-2.2 and tcl-dev 8.5.0-2.

See #612264 on that topic.

> This can be reproduced quite easily with these commands:
>
>   pbuilder --login
>   apt-get -y install file 
>   apt-get -y install tk-tile
>   apt-get -y install tcl-dev
>   file /usr/include/tcl
>   apt-get -y --purge remove tk-tile
>   apt-get -y --purge remove tcl-dev
>   apt-get -y install tcl-dev
>   apt-get -y install tk-tile
>   file /usr/include/tcl
>   dpkg -c /var/cache/apt/archives/tk-tile_0.8.2-2.2_amd64.deb | grep include/tcl
>   dpkg -c /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb | grep include/tcl
>
> In my opinion, regardless of the order in which these two packages are
> installed, dpkg should fail to install the second package, with an error
> message about /usr/include/tcl.

Policy (§6.6) actually documents this behavior:

  A directory will never be replaced by a symbolic link to a
  directory or vice versa; instead, the existing state (symlink or
  not) will be left alone and `dpkg' will follow the symlink if
  there is one.

Cheers,
       Sven

Message #15 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Sven Joachim <svenjoac@gmx.de>

To: Bart Martens <bartm@debian.org>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 19:44:34 +0200

retitle 681243 dpkg: warn when different packages have same file as directory and symlink
merge 406715 681243
thanks

On 2012-07-11 19:07 +0200, Sven Joachim wrote:

> On 2012-07-11 18:53 +0200, Bart Martens wrote:
>
>> Package: dpkg
>> Version: 1.16.7
>>
>> The file /usr/include/tcl is a directory or a symbolic link depending on the
>> order of installation of the packages tk-tile 0.8.2-2.2 and tcl-dev 8.5.0-2.
>
> See #612264 on that topic.
>
>> This can be reproduced quite easily with these commands:
>>
>>   pbuilder --login
>>   apt-get -y install file 
>>   apt-get -y install tk-tile
>>   apt-get -y install tcl-dev
>>   file /usr/include/tcl
>>   apt-get -y --purge remove tk-tile
>>   apt-get -y --purge remove tcl-dev
>>   apt-get -y install tcl-dev
>>   apt-get -y install tk-tile
>>   file /usr/include/tcl
>>   dpkg -c /var/cache/apt/archives/tk-tile_0.8.2-2.2_amd64.deb | grep include/tcl
>>   dpkg -c /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb | grep include/tcl
>>
>> In my opinion, regardless of the order in which these two packages are
>> installed, dpkg should fail to install the second package, with an error
>> message about /usr/include/tcl.
>
> Policy (§6.6) actually documents this behavior:
>
>   A directory will never be replaced by a symbolic link to a
>   directory or vice versa; instead, the existing state (symlink or
>   not) will be left alone and `dpkg' will follow the symlink if
>   there is one.

And changing it would break quite a few things.  However, emitting a
warning in such situations might be reasonable, see #406715.

Cheers,
       Sven

Message #26 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Bart Martens <bartm@debian.org>

To: Sven Joachim <svenjoac@gmx.de>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 19:18:43 +0000

On Wed, Jul 11, 2012 at 07:07:49PM +0200, Sven Joachim wrote:
> On 2012-07-11 18:53 +0200, Bart Martens wrote:
> > In my opinion, regardless of the order in which these two packages are
> > installed, dpkg should fail to install the second package, with an error
> > message about /usr/include/tcl.
> 
> Policy (§6.6) actually documents this behavior:
> 
>   A directory will never be replaced by a symbolic link to a
>   directory or vice versa; instead, the existing state (symlink or
>   not) will be left alone

OK.

>   and `dpkg' will follow the symlink if there is one.

I'm not convinced that this is a good choice.  Do you know where this choice
comes from ?

Regards,

Bart Martens

Message #31 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Bart Martens <bartm@debian.org>

To: Sven Joachim <svenjoac@gmx.de>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 19:47:00 +0000

On Wed, Jul 11, 2012 at 07:44:34PM +0200, Sven Joachim wrote:
> On 2012-07-11 19:07 +0200, Sven Joachim wrote:
> > On 2012-07-11 18:53 +0200, Bart Martens wrote:
> >> In my opinion, regardless of the order in which these two packages are
> >> installed, dpkg should fail to install the second package, with an error
> >> message about /usr/include/tcl.
> >
> > Policy (§6.6) actually documents this behavior:
> >
> >   A directory will never be replaced by a symbolic link to a
> >   directory or vice versa; instead, the existing state (symlink or
> >   not) will be left alone and `dpkg' will follow the symlink if
> >   there is one.
> 
> And changing it would break quite a few things.

Examples of those things ? How to reproduce those breakages ?

Regards,

Bart Martens

Message #36 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Sven Joachim <svenjoac@gmx.de>

To: Bart Martens <bartm@debian.org>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 22:14:26 +0200

On 2012-07-11 21:18 +0200, Bart Martens wrote:

> On Wed, Jul 11, 2012 at 07:07:49PM +0200, Sven Joachim wrote:
>> On 2012-07-11 18:53 +0200, Bart Martens wrote:
>> > In my opinion, regardless of the order in which these two packages are
>> > installed, dpkg should fail to install the second package, with an error
>> > message about /usr/include/tcl.
>> 
>> Policy (§6.6) actually documents this behavior:
>> 
>>   A directory will never be replaced by a symbolic link to a
>>   directory or vice versa; instead, the existing state (symlink or
>>   not) will be left alone
>
> OK.
>
>>   and `dpkg' will follow the symlink if there is one.
>
> I'm not convinced that this is a good choice.  Do you know where this choice
> comes from ?

You probably have to ask Ian Jackson; this behavior exists at least
since dpkg 1.0.0.

I suppose that some people used symlinks to relocate parts of their
directory hierarchy elsewhere to work around ENOSPC problems, and while
today it's much saner to use bind mounts to achieve that, those were not
available in 1995.

Cheers,
       Sven

Message #41 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Sven Joachim <svenjoac@gmx.de>

To: Bart Martens <bartm@debian.org>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 22:33:00 +0200

On 2012-07-11 21:47 +0200, Bart Martens wrote:

> On Wed, Jul 11, 2012 at 07:44:34PM +0200, Sven Joachim wrote:
>> On 2012-07-11 19:07 +0200, Sven Joachim wrote:
>> > On 2012-07-11 18:53 +0200, Bart Martens wrote:
>> >> In my opinion, regardless of the order in which these two packages are
>> >> installed, dpkg should fail to install the second package, with an error
>> >> message about /usr/include/tcl.
>> >
>> > Policy (§6.6) actually documents this behavior:
>> >
>> >   A directory will never be replaced by a symbolic link to a
>> >   directory or vice versa; instead, the existing state (symlink or
>> >   not) will be left alone and `dpkg' will follow the symlink if
>> >   there is one.
>> 
>> And changing it would break quite a few things.
>
> Examples of those things ?

Local admin has set up (say) /usr/share/doc as a symlink rather than a
plain directory.  Not really recommended, but according to your plan
they would not be able to unpack any package until they fix that.

Package foo converts (say) /usr/share/doc/foo into a symlink in the new
version.  There is no standard procedure for that right now[1], but the
sanest way is to do it in the postinst (look at the libpipeline-dev
package for an example).  With your proposal this becomes impossible,
and instead the package needs to rm -rf the directory in the preinst
instead, losing files if unpacking fails.

If other packages that share the same directory are involved, this
becomes even trickier.

> How to reproduce those breakages ?

Modify dpkg to fail in the above situations; this is left as an exercise
for you. ;-)

Cheers,
       Sven


1. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=583585

Message #46 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Bart Martens <bartm@debian.org>

To: Sven Joachim <svenjoac@gmx.de>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 20:57:07 +0000

On Wed, Jul 11, 2012 at 07:18:43PM +0000, Bart Martens wrote:
> On Wed, Jul 11, 2012 at 07:07:49PM +0200, Sven Joachim wrote:
> > On 2012-07-11 18:53 +0200, Bart Martens wrote:
> > > In my opinion, regardless of the order in which these two packages are
> > > installed, dpkg should fail to install the second package, with an error
> > > message about /usr/include/tcl.
> > 
> > Policy (§6.6) actually documents this behavior:
> > 
> >   A directory will never be replaced by a symbolic link to a
> >   directory or vice versa; instead, the existing state (symlink or
> >   not) will be left alone
> 
> OK.
> 
> >   and `dpkg' will follow the symlink if there is one.
> 
> I'm not convinced that this is a good choice.  Do you know where this choice
> comes from ?

Please have a look at this scenario :

  apt-get -y -t stable install tcl-dev
  apt-get -y -t stable install tk-tile
  apt-get -y -t unstable install tcl-dev

The first and second commands succeed.  The third command fails with this :

  Unpacking replacement tcl-dev ...
  dpkg: error processing /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb (--unpack):
   trying to overwrite '/usr/include/tcl', which is also in package tk-tile 0.8.2-2.1
  Preparing to replace tcl 8.4.16-2 (using .../archives/tcl_8.5.0-2_all.deb) ...
  update-alternatives: using /usr/bin/tclsh8.4 to provide /usr/bin/tclsh (tclsh) in auto mode
  Unpacking replacement tcl ...
  Errors were encountered while processing:
   /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb
  E: Sub-process /usr/bin/dpkg returned an error code (1)

So dpkg has initially installed /usr/include/tcl as a symbolic link, next dpkg
has tolerated the installation of tk-tile owning /usr/include/tcl as a
directory, and finally dpkg fails to upgrade the symbolic link because tk-tile
owns /usr/include/tcl as a directory.

Obviously dpkg should successfully upgrade the symbolic link.

The solution is, in my opinion, that dpkg fails to install the package owning
/usr/include/tcl as a directory when /usr/include/tcl is already installed as a
symbolic link.  Then there is no problem with upgrading the symbolic link.

But I'm still interested to know more about "changing it would break quite a
few things", see earlier on the bug report.

Regards,

Bart Martens

Message #51 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Sven Joachim <svenjoac@gmx.de>

To: Bart Martens <bartm@debian.org>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Wed, 11 Jul 2012 23:41:59 +0200

On 2012-07-11 22:57 +0200, Bart Martens wrote:

> Please have a look at this scenario :
>
>   apt-get -y -t stable install tcl-dev
>   apt-get -y -t stable install tk-tile
>   apt-get -y -t unstable install tcl-dev
>
> The first and second commands succeed.  The third command fails with this :
>
>   Unpacking replacement tcl-dev ...
>   dpkg: error processing /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb (--unpack):
>    trying to overwrite '/usr/include/tcl', which is also in package tk-tile 0.8.2-2.1
>   Preparing to replace tcl 8.4.16-2 (using .../archives/tcl_8.5.0-2_all.deb) ...
>   update-alternatives: using /usr/bin/tclsh8.4 to provide /usr/bin/tclsh (tclsh) in auto mode
>   Unpacking replacement tcl ...
>   Errors were encountered while processing:
>    /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb
>   E: Sub-process /usr/bin/dpkg returned an error code (1)
>
> So dpkg has initially installed /usr/include/tcl as a symbolic link, next dpkg
> has tolerated the installation of tk-tile owning /usr/include/tcl as a
> directory, and finally dpkg fails to upgrade the symbolic link because tk-tile
> owns /usr/include/tcl as a directory.

That's not actually an accurate description; dpkg has no way of knowing
which of the packages tcl-dev/stable and tk-tile, if any, ships
/usr/include/tcl as a directory.  The dpkg database just includes a list
of files for each package, not their type.

> Obviously dpkg should successfully upgrade the symbolic link.

And break the packages co-owning it?  This is not allowed unless you
specify --force-overwrite.  While directories are a shared resource,
symlinks to directories are not, unless they all point to the _same_
directory.  And since tcl-dev/unstable wants to change the target of the
symlink, the file conflict arises.

> The solution is, in my opinion, that dpkg fails to install the package owning
> /usr/include/tcl as a directory when /usr/include/tcl is already installed as a
> symbolic link.  Then there is no problem with upgrading the symbolic link.

But it breaks if the symbolic link has been installed by the sysadmin
rather than by a package.

> But I'm still interested to know more about "changing it would break quite a
> few things", see earlier on the bug report.

Did you receive my earlier mail <871uki6nsz.fsf@turtle.gmx.de> ?

Cheers,
       Sven

Message #56 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Bart Martens <bartm@debian.org>

To: Sven Joachim <svenjoac@gmx.de>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Thu, 12 Jul 2012 05:25:53 +0000

On Wed, Jul 11, 2012 at 11:41:59PM +0200, Sven Joachim wrote:
> On 2012-07-11 22:57 +0200, Bart Martens wrote:
> 
> > Please have a look at this scenario :
> >
> >   apt-get -y -t stable install tcl-dev
> >   apt-get -y -t stable install tk-tile
> >   apt-get -y -t unstable install tcl-dev
> >
> > The first and second commands succeed.  The third command fails with this :
> >
> >   Unpacking replacement tcl-dev ...
> >   dpkg: error processing /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb (--unpack):
> >    trying to overwrite '/usr/include/tcl', which is also in package tk-tile 0.8.2-2.1
> >   Preparing to replace tcl 8.4.16-2 (using .../archives/tcl_8.5.0-2_all.deb) ...
> >   update-alternatives: using /usr/bin/tclsh8.4 to provide /usr/bin/tclsh (tclsh) in auto mode
> >   Unpacking replacement tcl ...
> >   Errors were encountered while processing:
> >    /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb
> >   E: Sub-process /usr/bin/dpkg returned an error code (1)
> >
> > So dpkg has initially installed /usr/include/tcl as a symbolic link, next dpkg
> > has tolerated the installation of tk-tile owning /usr/include/tcl as a
> > directory, and finally dpkg fails to upgrade the symbolic link because tk-tile
> > owns /usr/include/tcl as a directory.
> 
> That's not actually an accurate description; dpkg has no way of knowing
> which of the packages tcl-dev/stable and tk-tile, if any, ships
> /usr/include/tcl as a directory.  The dpkg database just includes a list
> of files for each package, not their type.

This information is available at the time dpkg tries to install the packages.

dpkg -c /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb | grep /usr/include/tcl
dpkg -c /var/cache/apt/archives/tk-tile_0.8.2-2.2_amd64.deb | grep /usr/include/tcl/$
lrwxrwxrwx root/root         0 2011-02-06 14:30 ./usr/include/tcl -> tcl8.5
drwxr-xr-x root/root         0 2012-05-10 01:45 ./usr/include/tcl/

> 
> > Obviously dpkg should successfully upgrade the symbolic link.
> 
> And break the packages co-owning it?  This is not allowed unless you
> specify --force-overwrite.  While directories are a shared resource,
> symlinks to directories are not, unless they all point to the _same_
> directory.  And since tcl-dev/unstable wants to change the target of the
> symlink, the file conflict arises.

My point is that the real file conflict should be caught when dpkg tries to
install tk-tile, not when dpkg tries to upgrade the symbolic link.

> 
> > The solution is, in my opinion, that dpkg fails to install the package owning
> > /usr/include/tcl as a directory when /usr/include/tcl is already installed as a
> > symbolic link.  Then there is no problem with upgrading the symbolic link.
> 
> But it breaks if the symbolic link has been installed by the sysadmin
> rather than by a package.

I understand now about symbolic links added by the sysadmin as local changes,
and about the choice made in the past that dpkg follows those symbolic links as
mentioned in debian-policy.  However, dpkg can see the difference between those
locally added symbolic links and symbolic links owned by installed packages.
The solution is, in my opinion, that dpkg fails to install the package owning
/usr/include/tcl as a directory when /usr/include/tcl is already installed as a
symbolic link owned by an already installed package.  Then there is no problem
with upgrading the symbolic link.

> 
> > But I'm still interested to know more about "changing it would break quite a
> > few things", see earlier on the bug report.
> 
> Did you receive my earlier mail <871uki6nsz.fsf@turtle.gmx.de> ?

I haven't saved the message-id's but I have now read all messages that are
logged on the bug report.  I think that I have fully understood everything
written so far.

The phrase "changing it would break quite a few things" was about the part from
debian-policy you quoted.  That is clear now.  Do you see other things break
when dpkg would be modified to fail to install a package owning a directory
that is already a symbolic link owned by an already installed package ? Same
question for the reverse scenario : do you see other things break when dpkg
would be modified to fail to install a package owning a symbolic link that is
already a directory owned by an already installed package ?

Regards,

Bart Martens

Message #61 received at 681243@bugs.debian.org (full text, mbox, reply):

From: Sven Joachim <svenjoac@gmx.de>

To: Bart Martens <bartm@debian.org>

Cc: 681243@bugs.debian.org

Subject: Re: Bug#681243: dpkg: fails to fail when different packages have same file as directory and symlink

Date: Thu, 12 Jul 2012 20:58:04 +0200

On 2012-07-12 07:25 +0200, Bart Martens wrote:

> On Wed, Jul 11, 2012 at 11:41:59PM +0200, Sven Joachim wrote:
>> 
>> That's not actually an accurate description; dpkg has no way of knowing
>> which of the packages tcl-dev/stable and tk-tile, if any, ships
>> /usr/include/tcl as a directory.  The dpkg database just includes a list
>> of files for each package, not their type.
>
> This information is available at the time dpkg tries to install the packages.

But it is not stored in the database, so the information is only
available for the package dpkg currently tries to install.

> dpkg -c /var/cache/apt/archives/tcl-dev_8.5.0-2_all.deb | grep /usr/include/tcl
> dpkg -c /var/cache/apt/archives/tk-tile_0.8.2-2.2_amd64.deb | grep /usr/include/tcl/$
> lrwxrwxrwx root/root         0 2011-02-06 14:30 ./usr/include/tcl -> tcl8.5
> drwxr-xr-x root/root         0 2012-05-10 01:45 ./usr/include/tcl/

The second and third line should be transposed here, I suppose?  Anyway,
at the time dpkg installs tcl-dev 8.5.0-2, it cannot know that tk-tile
had wanted to install /usr/include/tcl as a directory.

>> > Obviously dpkg should successfully upgrade the symbolic link.
>> 
>> And break the packages co-owning it?  This is not allowed unless you
>> specify --force-overwrite.  While directories are a shared resource,
>> symlinks to directories are not, unless they all point to the _same_
>> directory.  And since tcl-dev/unstable wants to change the target of the
>> symlink, the file conflict arises.
>
> My point is that the real file conflict should be caught when dpkg tries to
> install tk-tile, not when dpkg tries to upgrade the symbolic link.

As I said, this breaks existing systems, possibly up to the point where
they might not be able to unpack any package at all.

> I understand now about symbolic links added by the sysadmin as local changes,
> and about the choice made in the past that dpkg follows those symbolic links as
> mentioned in debian-policy.  However, dpkg can see the difference between those
> locally added symbolic links and symbolic links owned by installed packages.

No, it cannot.  When the sysadmin creates a symlink (say, /foo) to a
directory and a package which ships /foo as an empty directory is
installed later, the outcome is identical to what it would have been if
said package had shipped /foo as a symlink in the first place.

> The solution is, in my opinion, that dpkg fails to install the package owning
> /usr/include/tcl as a directory when /usr/include/tcl is already installed as a
> symbolic link owned by an already installed package.

You would also have to forbid it if it's a symlink on the system not
owned by any package, otherwise your next sentence...

> Then there is no problem with upgrading the symbolic link.

...would not actually be true.

> The phrase "changing it would break quite a few things" was about the part from
> debian-policy you quoted.  That is clear now.  Do you see other things break
> when dpkg would be modified to fail to install a package owning a directory
> that is already a symbolic link owned by an already installed package ?

I suppose the /run transition would have been quite a bit harder then,
since it would have involved Conflicts with every package that ships a
/var/run directory.

> Same
> question for the reverse scenario : do you see other things break when dpkg
> would be modified to fail to install a package owning a symbolic link that is
> already a directory owned by an already installed package ?

Again the /var/run transition; plus it is potentially much harder to
work around in preinst scripts (you may have to rm -rf whole directory
trees rather than just a symlink).

Cheers,
       Sven

Send a report that this bug log contains spam.