Solved cannot make install clean in jail

fullauto2012

fullauto2012

After some fighting I finally got ezjail to work.
Some overview before I hit you all up with a few issues and questions.

Running FreeBSD 10.1 on a VBox for testing purposes.

Code: 
root@freebsd_vm01:/usr/home/tim.falardeau # cat /etc/rc.conf
hostname="freebsd_vm01"
ifconfig_em0="inet 192.168.1.50 netmask 255.255.255.0"
ifconfig_em0_alias0="inet 192.168.1.51 netmask 255.255.255.0"
defaultrouter="192.168.1.1"

sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
ezjail_enable="YES"

Code: 
root@freebsd_vm01:/etc # cat /etc/resolv.conf

nameserver 192.168.1.1
nameserver 8.8.4.4

Code: 
root@freebsd_vm01:/usr/local/etc # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
  ether 08:00:27:34:59:b2
  inet 192.168.1.50 netmask 0xffffff00 broadcast 192.168.1.255
  inet 192.168.1.51 netmask 0xffffffff broadcast 192.168.1.51
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
root@freebsd_vm01:/usr/local/etc #

Code: 
root@freebsd_vm01:/usr/local/etc/ezjail # cat /usr/local/etc/ezjail/www
# To specify the start up order of your ezjails, use these lines to
# create a Jail dependency tree. See rcorder(8) for more details.
#
# PROVIDE: standard_ezjail
# REQUIRE:
# BEFORE:
#

export jail_www_hostname="www"
export jail_www_ip="192.168.1.51"
export jail_www_rootdir="/usr/jails/www"
export jail_www_exec_start="/bin/sh /etc/rc"
export jail_www_exec_stop=""
export jail_www_mount_enable="YES"
export jail_www_devfs_enable="YES"
export jail_www_devfs_ruleset="devfsrules_jail"
export jail_www_procfs_enable="YES"
export jail_www_fdescfs_enable="YES"
export jail_www_image=""
export jail_www_imagetype=""
export jail_www_attachparams=""
export jail_www_attachblocking=""
export jail_www_forceblocking=""
export jail_www_zfs_datasets=""
export jail_www_cpuset=""
export jail_www_fib=""
export jail_www_parentzfs=""
export jail_www_parameters=""
export jail_www_post_start_script=""
export jail_www_retention_policy=""

The jail 'www' is running.
Code: 
root@freebsd_vm01:/usr/local/etc # jls
  JID  IP Address  Hostname  Path
  3  192.168.1.51  www  /usr/jails/www

Code: 
root@freebsd_vm01:/usr/local/etc # ping 192.168.1.51
PING 192.168.1.51 (192.168.1.51): 56 data bytes
64 bytes from 192.168.1.51: icmp_seq=0 ttl=64 time=0.074 ms
64 bytes from 192.168.1.51: icmp_seq=1 ttl=64 time=0.104 ms
64 bytes from 192.168.1.51: icmp_seq=2 ttl=64 time=0.108 ms
64 bytes from 192.168.1.51: icmp_seq=3 ttl=64 time=0.107 ms
64 bytes from 192.168.1.51: icmp_seq=4 ttl=64 time=0.106 ms
^C
--- 192.168.1.51 ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.074/0.100/0.108/0.013 ms

>ezjail-admin console www:

Code: 
root@www:/etc # cat rc.conf
ifconfig_em0="inet 192.168.1.51 netmask 255.255.255.0"
default_router="192.168.1.1"

Code: 
root@www:/etc # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
  ether 08:00:27:34:59:b2
  inet 192.168.1.51 netmask 0xffffffff broadcast 192.168.1.51
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
root@www:/etc #

I can ssh to my other server
Code: 
root@www:/etc # ssh 192.168.1.10
Password for root@kif:

But I cannot build any ports to populate the jail!

Code: 
root@www:/etc # cd /usr/ports/www/apache24
root@www:/usr/ports/www/apache24 # make install clean
===> Building/installing dialog4ports as it is required for the config dialog
===>  Cleaning for dialog4ports-0.1.5_2
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
===>  dialog4ports-0.1.5_2 depends on file: /usr/local/sbin/pkg - not found
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
=> pkg-1.8.3.tar.xz doesn't seem to exist in /var/ports/distfiles/.
=> Attempting to fetch http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz
fetch: http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.us-east.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.us-east.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.eu.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.eu.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.us-west.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.us-west.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://mirror.shatow.net/freebsd/pkg/pkg-1.8.3.tar.xz
fetch: http://mirror.shatow.net/freebsd/pkg/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/pkg-1.8.3.tar.xz
fetch: http://distcache.FreeBSD.org/ports-distfiles/pkg-1.8.3.tar.xz: No address record
=> Couldn't fetch it - please try to retrieve this
=> port manually into /var/ports/distfiles/ and try again.
*** Error code 1

Stop.
make[5]: stopped in /basejail/usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[4]: stopped in /basejail/usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[3]: stopped in /basejail/usr/ports/ports-mgmt/dialog4ports
*** Error code 1

Stop.
make[2]: stopped in /basejail/usr/ports/ports-mgmt/dialog4ports
===> Options unchanged
===>  apache24-2.4.20_1 depends on file: /usr/local/sbin/pkg - not found
===>  License BSD2CLAUSE accepted by the user
=> pkg-1.8.3.tar.xz doesn't seem to exist in /var/ports/distfiles/.
=> Attempting to fetch http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz
fetch: http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.us-east.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.us-east.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.eu.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.eu.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.us-west.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz
fetch: http://distcache.us-west.FreeBSD.org/local-distfiles/portmgr/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://mirror.shatow.net/freebsd/pkg/pkg-1.8.3.tar.xz
fetch: http://mirror.shatow.net/freebsd/pkg/pkg-1.8.3.tar.xz: No address record
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/pkg-1.8.3.tar.xz
fetch: http://distcache.FreeBSD.org/ports-distfiles/pkg-1.8.3.tar.xz: No address record
=> Couldn't fetch it - please try to retrieve this
=> port manually into /var/ports/distfiles/ and try again.
*** Error code 1

Stop.
make[2]: stopped in /basejail/usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[1]: stopped in /basejail/usr/ports/www/apache24
*** Error code 1

Stop.
make: stopped in /basejail/usr/ports/www/apache24
root@www:/usr/ports/www/apache24 #

I am very confused. I have always used cd /usr/ports/.../.../; make install clean to install things. I don't mind having traceroute, ping, and such. But, without being able to get to www.freebsd.org for ports and such, how am I supposed to do this?

I understand that /usr/jails/basejail has the ports dir used for all jails commonly, and that each jail issues a mount_nullfs to mount the common ports dir from inside the host as by the fstab.<jail.name> file, which is being done:

Code: 
root@freebsd_vm01:/etc # cat /etc/fstab.www
/usr/jails/basejail /usr/jails/www/basejail nullfs ro 0 0
root@freebsd_vm01:/etc #

But, what am I missing? is not being able to route OUT from the jail a feature, and not something I should fix? How do I install ports without being able to route to www.FreeBSD.org? Is there a way I can turn on ALL traffic, ICMP included, to the jails and just control things with PF the way I am accustomed? And finally, I keep getting emails from a Nigerian Prince who wants to send me $1 million, and all I have to do is send him a cashiers check for $2k... Should I do it?
 
Routing is not the issue.
Code: 
fetch: http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz: No address record
"No address record" means you don't have proper DNS configured.
 
Specifically, enter the local nameserver in /etc/resolv.conf. Also a good idea to edit /etc/hosts with the jail hostname.
 
wblock@ said:
Specifically, enter the local nameserver in /etc/resolv.conf. Also a good idea to edit /etc/hosts with the jail hostname.
Click to expand...

Did both of those... Still having the same issue...
Do you think it's possible the fact that the host is a Vbox is the issue?
 
I did do it on the jail. Host doesn't have any problems.

Code: 
root@www:/usr/ports/www/apache24 # cat /etc/hosts
# $FreeBSD: releng/10.1/etc/hosts 109997 2003-01-28 21:29:23Z dbaker $
192.168.1.50  freebsd_vm01
root@www:/usr/ports/www/apache24 #

Code: 
root@www:/usr/ports/www/apache24 # cat /etc/resolv.conf
nameserver 192.168.1.1
nameserver 8.8.4.4
root@www:/usr/ports/www/apache24 #

Is there maybe a sysctl variable that needs to be set?
I've never been very good remembering those...
 
Nothing needs to be set for this to work.

What does host www.freebsd.org output in the jail?
 
SirDice said:
Nothing needs to be set for this to work.

What does host www.freebsd.org output in the jail?
Click to expand...
Code: 
root@www:~ # host www.freebsd.org
www.freebsd.org is an alias for wfe0.ysv.freebsd.org.
wfe0.ysv.freebsd.org has address 8.8.178.110
wfe0.ysv.freebsd.org has IPv6 address 2001:1900:2254:206a::50:0
wfe0.ysv.freebsd.org mail is handled by 0 .
root@www:~ #
wblock@ said:
Weird things will happen if the jail IP address is being used by something else already, too.
Click to expand...

Nothing else is on 192.168.1.51
I reserved 192.168.1.50-100 for Vbox testing.
 
Ok, so it looks like name resolving works. Try the following in the jail:
host files.etoilebsd.net
This should output some IP addresses. Then try:
fetch http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz
 
Dear fullauto2012,
the only thing I can see that a cloned loopback interface is not configured in your /etc/rc.conf. http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails-ezjail.html says
Code: 
Installing ezjail consists of adding a loopback interface for use in jails, installing the port or package, and enabling the service.
I am not sure if the cloned loopback interface is necessary or not. On my host I have as part of /etc/rc.conf
Code: 
# Required for jails
ezjail_enable="YES"
cloned_interfaces="lo1"
ifconfig_lo1="inet 10.0.0.254 netmask 255.255.255.0"
gateway_enable="YES"
I hope this is helpful and not misleading. I should mention that my host is a real machine and not virtual. I do not know if this can make a difference. Have success!
 
SirDice said:
Ok, so it looks like name resolving works. Try the following in the jail:
host files.etoilebsd.net
This should output some IP addresses. Then try:
fetch http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz
Click to expand...

Code: 
root@www:~ # host files.etoilebsd.net
files.etoilebsd.net is an alias for etoilebsd.net.
etoilebsd.net has address 178.32.217.76
etoilebsd.net has IPv6 address 2001:41d0:8:db4c::1
etoilebsd.net mail is handled by 20 fb.mail.gandi.net.
etoilebsd.net mail is handled by 10 ivaldir.etoilebsd.net.
root@www:~ #

chrbr said:
# Required for jails ezjail_enable="YES" cloned_interfaces="lo1" ifconfig_lo1="inet 10.0.0.254 netmask 255.255.255.0" gateway_enable="YES"
Click to expand...

I did this once you suggested it.


Code: 
root@www:/usr/ports/www/apache24 # make install clean
===> Building/installing dialog4ports as it is required for the config dialog
===>  Cleaning for dialog4ports-0.1.5_2
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
===>  dialog4ports-0.1.5_2 depends on file: /usr/local/sbin/pkg - not found
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
=> pkg-1.8.3.tar.xz doesn't seem to exist in /var/ports/distfiles/.
=> Attempting to fetch http://files.etoilebsd.net/pkg/pkg-1.8.3.tar.xz
pkg-1.8.3.tar.xz  42% of 1897 kB  621 kBps 00m02s
fetch: transfer interrupted

root@www:/usr/ports/www/apache24 #

Seems to be working now.... Much thanks...

One more question.
Do I just keep adding clones for each jail now?
 
fullauto2012 said:
Do I just keep adding clones for each jail now?
Click to expand...
The jails can attach to one cloned interface of the host. Please see as below an extract out of my jails configurations
Code: 
# cat * | grep _ip | grep -v #
export jail_bastel_ip="lo1|10.0.0.4,bge0|192.168.0.1"
export jail_box_ip="lo1|10.0.0.2"
export jail_fox_ip="lo1|10.0.0.3"
export jail_www_ip="lo1|10.0.0.1"
Just my "bastel" jail has a direct connection to my external network interface. All others are communicating via lo1. I think the explanation of kpa in Thread 56354 improves the understanding about cloned inerfaces, at least I hope it did that for me :).
 
One last question...
How do you make jails able to connect to each other?
I have two jails now, www (51) and mysql (52).
When in the mysql jail and I try to ssh to www, it logs me into the host machine (50)
Or is that an anomaly of jails, that you cannot ssh into them directly from other jails because the host system intercepts the call?
 
fullauto2012 said:
When in the mysql jail and I try to ssh to www, it logs me into the host machine (50)
Click to expand...
Bind all services on the host to a specific host IP address. By default services will bind to all IP addresses, including those belonging to jails.
 
chrbr said:
I am not sure, I have not tried that up to now.

I would not wonder if any jail to jail communication would be open by default, it would count as a security issue. But I am not sure.
Click to expand...

The jails themselves have nothing to control TCP/IP traffic between different jails with the exception of IP address visibility access that usually limits the access to just one IP address from the host and excludes the localhost. Everything else has to be handled outside them by the packet filter on the host and separation of the jails to different interfaces.
 
Dear kpa, thank you for the explanation. I really appreciate to learn from the advises.
fullauto2012 said:
How do you make jails able to connect to each other?
I have two jails now, www (51) and mysql (52).
When in the mysql jail and I try to ssh to www, it logs me into the host machine (50)
Or is that an anomaly of jails, that you cannot ssh into them directly from other jails because the host system intercepts the call?
Click to expand...
I have tried that now. The jails I have are
Code: 
$ ezjail-admin list
STA JID  IP  Hostname  Root Directory
--- ---- --------------- ------------------------------ ------------------------
DR  1  10.0.0.1  www  /usr/jails/www
DR  2  10.0.0.3  fox  /usr/jails/fox
DSN N/A  10.0.0.2  box  /usr/jails/box
DR  3  10.0.0.4  bastel  /usr/jails/bastel
  3  bge0|192.168.0.1
The short form as my user ssh bastel let me ssh to the "bastel" jail because it is configured like that. From there I can hop through the jails as user chris using ssh 10.0.0.1 to ssh 10.0.0.4. Each exit leads the way back. The jail "box" is currently not running.

It does not work as root per default because it is not enabled in /etc/ssh/sshd_config. I kept the default. fullauto2012, have you tried to ssh as user or as root? May be you already know that already, you can specify the target account as ssh "www-user"@192.168.1.51. Otherwise the current user name is applied.
 
You must log in or register to reply here.
Back
Top