Basic firewalld

As someone who dreaded having to interact with the esoteric networking gatekeeper that was iptables, firewalld presented an opportunity for mere mortals to feel like more of a badass when crafting ingress rules. Although firewalld manages iptables, some abstraction is most welcome, if incomplete. For example, playing in the firewalld arena will only handle ingress traffic. If you need granular control over egress traffic, you’ll still need to dive into iptables, but you’ll triage these through firewalld’s so-called rich rules.

firewalld sees fragmented adoption across various distributions; maybe because firewalld isn’t the only netfilter abstraction in town, or because we all want to be different. Most distributions offer firewalld through their default repositories even if it’s not the existing sheriff, so if you want to run it instead of whatever else was on offer, you’ll want to remove the original program first. Most every RedHat-based distribution will be running firewalld as a default. Ubuntu-based shite will likely have ufw or something else to that effect (ergo, if you’re forced at gunpoint to use any of that garbage, get yourself firewalld immediately).

Contained within firewalld are the concept of zones. Each zone encapsulates a different set of rules that are logically associated to the zone itself. Not only are there a decent handful of default zones – which are more than sufficient for garden-variety use cases – but you have the ability to create and delete other zones (You’re unable to delete any of the stock options. I tried about ninety times.). Each zone can be applied to a particular interface, be it physical or virtual. The rules within each of these zones will dictate how ingress traffic is handled. For example, you can configure a zone to disallow ICMP traffic to the host, or drop all traffic other than a select handful of services.

As with zones, firewalld offers a plethora of default services that can be used. Services are a collection of colloquial protocol/port mappings consolidated under an easy to understand identifier. They’re intended to save time with building zones, by being readily available to any zone that wants them. You can also add or delete custom services, just as you can with zones. For example, the firewalld service http will map to tcp/80, https will map to tcp/443, ssh will map to tcp/22, and so on.

And this is essentially all you’ll need to know in order to get some reliable mileage out of your firewalld installation. This says nothing about the details of rich rules, IPSets, or Helpers, but these are more advanced topics that can be understood by reading the official firewalld documentation. Think of this document as a way to whet your appetite and help you play with a tool. Note that going forward, all commands displayed will be assuming that you’re running a RedHat-based distribution that leverages systemd.

To start, you can ensure that firewalld is running by querying systemd:

systemctl status firewalld

And obviously, you can toggle the state of the daemon by using one of the following:

systemctl start firewalld
systemctl stop firewalld

You can use the reload command as well for forcing configuration changes, but there’s an alternative method to this which we’ll cover momentarily.

Interfacing with firewalld is facilitated by either the terminal command firewall-cmd or by the GUI client firewall-config (which can also partner with firewall-applet, assuming you’re running a GUI). This document will focus only on the terminal interface, especially since most enterprise production servers will be operating headless.

The obligatory commands are available for your typing pleasure:

firewall-cmd --version
firewall-cmd --help
man firewall-cmd

Trust me, the man pages for this program are very good.

Now, although the daemon may be running, the firewall may be in a state where it’s not enforcing. You can query the current state of the firewall using the following:

firewall-cmd --state

You can determine which zones are active (i.e. a binding to an interface that has an active connection).

firewall-cmd --get-active-zones

If you wish to see the zone that’s associated with a particular interface:

firewall-cmd --get-zone-of-interface=<ifname>

You can get the names of your interfaces by using either of the following:

nmcli c
ip addr sh

A list of all the zones known to firewalld can be obtained.

firewall-cmd --get-zones

The same can be done to get a complete listing of all hardcoded services that can be used in zone configurations.

firewall-cmd --get-services

Now that you know how to see zones, regardless if they’re active or passive, you’ll want to see the configuration of the zone itself.

firewall-cmd --zone=<zonename> --list-all

Again, the name of a zone can be obtained by either listing all of the zones or determining which zone is associated with your active network interface.

A similar breakdown for services is available. Sometimes a service can encapsulate multiple ports or other targets, so knowing what the service identifier is referencing is important. For example, if you want to know what the service ssh contains, you’ll issue the following command:

firewall-cmd --info-service=ssh

Now that we can obtain some rudimentary information about both zones and their services, we can move forward modifying existing zones. However, there is still a bit more to know before going too far down the rabbit hole.

Aside from services, there are a few other basic properties of zones that you need to pay attention to, especially when considering which zone to use or if you’re designing your own.

Every zone has a target. The target is effectively a so-called next-hop for the packet after applying the filter rules in the current zone. There are three targets available, and any given zone can only have one target.

ACCEPT – Any packet not matching any rule is permitted.
%%REJECT%% – Any packet not matching any rule is rejected.
DROP – Any packet not matching any rule is dropped.

In practise, what this means is that if a zone has a target of ACCEPT, virtually all packets are permitted. %%REJECT%% and DROP will deny packets based on rules, but a denial under the former will trigger an ICMP response back to the source, whereas the latter will simply discard the packet with no response. Ergo, under a DROP target, it might not be obvious to clients if something is amiss, and the absence of diagnostic messages could make troubleshooting for lower-tier support more difficult than it need be otherwise.

Next are ICMP Blocks. ICMP provides a few neat features for querying devices on your network. One of the most common ICMP functions is ping, which is used to determine host visibility (which, in reality, is a somewhat erroneous assumption once you understand how the service is classed). However, being able to obtain this kind of information may not be desirable in certain contexts. For example, while you may want certain ports on an infrastructure server exposed, you may also not want the server to be pingable by any random associate. And while there are definitely more robust and reliable ways of achieving this goal, for the sake of this discussion, we’ll say that we simply want to disallow pinging.

ICMP Blocks under firewalld are broken into two categories: individual and masked. To understand this, one need look no further than the zone information for the default zone public. As default, icmp-block-inversion is no, and there are no individual icmp-blocks. Effectively, this permits all ICMP traffic. Now, we have two options here for blocking ICMP traffic. We can either add individual ICMP services to the zone, or we can develop a permutation that utilizes both individual blocks and/or a block inversion. The block inversion simply takes the configured ICMP Blocks and flips them around, or inverts them. Thus, if we add no individual ICMP services but add an ICMP Block Inversion, we are now blocking all ICMP services. If we add an ICMP Block Inversion as well as specific ICMP services, we are now permitting ONLY the specified services.

That sounds like quite a bit, but we can summarise it as thus:

Basic building blocks of zones are targets, services, ICMP services, and ICMP Block Inversions. Knowing how to manipulate these will go a long way.

This is a gross over-simplification, but knowledge here can make all the difference in most cases.

One last thing regarding changes to firewalld. Any changes issued are as default memory-resident only. Unless explicitly committed, changes will be wiped when the system goes down. Adding the –permanent option to your commands will ensure that modifications survive power cycles.

Let’s walk through the process of creating a new zone called ZONE_OF_POWER. It’s target will be %%REJECT%%, it’ll permit SSH, HTTP, HTTPS, and NTP traffic, and deny all ICMP except for ping. We can accomplish this with the following:

firewall-cmd --permanent --new-zone=ZONE_OF_POWER
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-service=ssh
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-service=http
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-service=https
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-service=ntp
firewall-cmd --permanent --zone=ZONE_OF_POWER --set-target=%%REJECT%%
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-icmp-block={echo-request,echo-reply}
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-icmp-block-inversion
firewall-cmd --reload

Time for a breakdown.

First, notice how all of the statements issued have the –permanent option in them. This is to ensure that our changes are rendered gospel by the firewalld overlords.

The first statement creates a new zone called ZONE_OF_POWER. Zones in firewalld are actually structured XML files, but we’re not going to dive into those here.

The following four statements add ssh, http, https, and ntp services to our new zone. This means that ingress traffic matching these services will be permitted to pass. Other kinds of traffic will be passed to the %%REJECT%% chain.

Next, we assign the %%REJECT%% target to our new zone.

Following that, we add two ICMP services, echo-request and echo-reply. These two form the foundation of a ping, and if we stopped here, we’d be instructing firewalld to block pings and permit everything else, which is not precisely what we set out to do.

Finally, we add an ICMP Block Inversion. This means that we take our current ICMP Blocks and flip them. With this added, we’re now permitting only ping requests and denying everything else.

By the way, as was mentioned before about both zones and services, you can obtain a full list of ICMP types that are stock to firewalld, so you know what to add or remove when dealing with them:

firewall-cmd --get-icmptypes

It’s also possible to add your own ICMP types, but this is beyond the scope here.

The very last statement will force firewalld to reload its configurations. This will permit you to assign ZONE_OF_POWER to an available interface. Speaking of which, if you want to add an interface to this new zone, you’d do it like this:

firewall-cmd --permanent --zone=ZONE_OF_POWER --add-interface=<ifname>

Note that this may throw an error, depending upon how angry DBus is on that particular day. I actually still don’t know why it happens, but occasionally you’ll get a quark error when attempting to place an interface into a new zone, requiring you to reboot the host to resolve it (at least my current understanding makes this the path of least resistance). If anyone has any ideas, filling me in would be great.

Finally, let’s talk about custom services. Custom services are useful if you plan on using custom ports or migrating existing services to non-standard ports. For example, if you decide to have SSH operating on port 2500 instead of 22, you’ll likely want to create a new service. While you might be able to modify the existing service definition, it’s probably best to create a whole new service for the sake of clarity and maintenance.

The following statements will create a new service called CUSTOM_SSH and add TCP port 2500 to it. Then, we’ll remove the existing ssh service from our custom zone from above and replace it with the new CUSTOM_SSH service.

firewall-cmd --permanent --new-service=CUSTOM_SSH
firewall-cmd --permanent --service=CUSTOM_SSH --add-port=2500/tcp
firewall-cmd --reload
firewall-cmd --permanent --zone=ZONE_OF_POWER --remove-service=ssh
firewall-cmd --permanent --zone=ZONE_OF_POWER --add-service=CUSTOM_SSH
firewall-cmd --reload

The first statement will tell firewalld that we want to create a new service definition called CUSTOM_SSH. Then we want to add the TCP port 2500 to that service definition. We’ll then reload the daemon so that we have the service available for distribution to other objects. Next, we’ll remove the existing ssh service, and then add the new CUSTOM_SSH service. Once we reload, the firewall should be ready to start permitting SSH traffic on TCP port 2500.*

  • – There are several peripheral caveats with this particular example. First, sshd needs configured to listen on port 2500. Second, if your computer is running SELinux, you’ll need to manipulate it to permit SSH traffic on a non-standard port. Both of these configurations are beyond the scope of this document.

Having finished this document, you should be able to start using firewalld in a basic, if not isolated, sense.

unsplash-logoBit Cloud

RHEL/CentOS – Configuring a Local Repository Server

I haven’t made a decent technology post in some time, let alone one about my beloved Linux. To rectify that travesty, this post comes at a time to end said drought and to share with my Linux friends how to accomplish the goal outlined by the title. I do often see a fair number of questions as to how to get a local repository setup, but the contexts always vary a little by virtue of distribution as well as delivery mechanism. The objective here is to get up and running with as little hassle as possible. Our target distribution will be RHEL/CentOS 7 with delivery facilitated via FTP.

Getting right to it, there are six key points we need to address:

  1. Configuration of the FTP server
  2. Location for the software packages that will compose the repository(-ies)
  3. Creation of the repository metadata
  4. Firewall configuration
  5. SELinux configuration
  6. Exposure

FTP Server

Both RHEL and CentOS will by default install vsftpd as the FTP server of choice. This can be installed during the package selection phase of Anaconda by first selecting the Infrastructure Server profile and then the FTP Server group. You can install several other groups as well, but we only focus on this one here. Please note that if you intend on using another FTP package, you’ll want to skip some parts of this tutorial since it’s assumed that you are going to use vsftpd. If during the installation process you fail to install this package, simply install it post-installation with yum:

yum install vsftpd

The nice thing about installing it from Anaconda is that the default vsftpd configuration file (/etc/vsftpd/vsftpd.conf) is geared toward a public anonymous context with maybe only some slight modifications being required. Removing all of the comments from the file for brevity, these are the options that are used on the server I’ve configured:

anonymous_enable=YES
local_enable=YES
write_enable=NO
local_umask=022
xferlog_enable=YES
connect_from_port_20=YES
xferlog_std_format=YES
listen=YES
listen_ipv6=NO
pam_service_name=vsftpd
tcp_wrappers=YES

If you have to make any changes to this file, be sure to restart the daemon after writing said changes:

systemctl restart vsftpd

File Location

The repository files will be kept in a custom location, not in /var/ftp/pub – the default landing directory for anonymous. For this example, the files will be stored in /opt/repos. This will make it easier if we want to have multiple repositories on the same server. In other words, we could have a directory for the base CentOS packages – copied from the resource DVD – and custom packages for your company under another directory. Hypothetically, this leaves us with the following directory structures:

/opt/repos/centos/7/dvd/packages
/opt/repos/company/packages

Finally, you’ll want to change the permissions for the /opt/repos directory and its contents:

chmod -R 755 /opt/repos

Create Repositories

Next you’ll want to populate your newly created directory structures with the packages that will comprise the repository. Keeping with our example directories, the first will need to be copied from the CentOS Resource DVD. This can be accomplished by mounting the ISO/DVD and copying everything out of the Packages directory in the root of the mount. If you have them stored somewhere else, you’ll need to get them onto the server by other means (SCP, NFS, SMB, etc…). The same principle applies for the custom packages that you’ll be putting in the second directory. Ultimately, your storage and security strategy will dictate where the files are actually stored at and how they’re placed in those two aforementioned directories.

If you haven’t done so already, you’ll want to install the createrepo package. This package will automate the creation of the metafiles required for both identifying and advertising the manifest of a repository. Once you have the package installed, you’ll want to use it to create the repository for each of these directories. Createrepo takes a path as an argument. It also takes other options, but for this example, it’s satisfactory enough to omit them and provide only the directory. When you do this, do not inlcude the packages portion of the path. Instead, you’ll provide the path up to that point. Obviously, your current working directory will determine what you have to provide to createrepo. It’ll most likely look like one of the following forms:

createrepo /opt/repos/centos7/dvd
createrepo .
createrepo ../

Use whichever one you want, so long as the path is correct. The larger the number of packages createrepo has to parse, the longer it’ll take to process. As you might notice, parsing all of the CentOS Resource DVD packages will take some time (there’s almost 10,000 packages).

When you create the repositories in this manner, you’re essentially creating a static repository. If any of the packages in there change, you’ll need to issue a command to createrepo to update the repository manifest. Or you can make createrepo occasionally sync with a mirror, but these configurations are outside the immediate scope of this document.

Once the packages are in place and the repositories created, you’ll need to get them to the point of being exposed by the FTP server. Because the default public anonymous location for vsftpd is /var/ftp/pub, and because vsftpd doesn’t handle following symlinks very well (or not at all), you’ll want to use a bind mount to get the directory over there. The basic gist of a bind mount is to mount a portion of the filesystem to another portion of the filesystem. So if you perform a listing command on /var/ftp/pub prior to the bind mount, you’ll likely get nothing back since that directory doesn’t (or at least shouldn’t) contain anything after a fresh installation. If you perform a bind mount:

mount --bind /opt/repos /var/ftp/pub

and then list directory contents:

ls /var/ftp/pub

you’ll get returned two directories. Note that this doesn’t survive reboots unless you add the mount to fstab:

/opt/repos /var/ftp/pub xfs defaults,bind 0 0

If you don’t do this, don’t be shocked if after a reboot you can’t see anything on your FTP server since the mount will be disconnected.

Firewall Configuration

There’s a pretty good chance that your network interface is going to have the public zone from firewalld applied to it during the installation process. Even so, this zone likely won’t have the FTP server whitelisted, so you’ll need to check to see if it is and react accordingly.

First, check what name was given to the interface (henceforth referred to as ifname). Hopefully, it wasn’t given something unpredictable. You won’t have to do this if you explicitly name your interfaces during installation.

ip addr sh

Once you have the interface name, check to see which zone was applied to it through firewalld (henceforth referred to as zname):

firewall-cmd --get-zone-of-interface=ifname

Now, use this zone name to see what kinds of traffic are permitted or disallowed:

firewall-cmd --zone=zname --list-all

Most likely, if this is the default public zone, you’ll only see the services ssh and dhcpv6-client. The key here though is that if you don’t see ftp in the list of services, you’re going to need to add it.

firewall-cmd --permanent --zone=zname --add-service ftp

If during the initial configuration of vsftpd you elected to run FTP through a non-standard port, you’ll need to make the appropriate accommodations:

firewall-cmd --permanent --zone=zname --add-port port#/protocol

Once that’s done, reload firewalld rules:

firewall-cmd --reload

SELinux Configuration

At this point, you should be able to run an external port scan and see your FTP port open. However, it’s unlikely that you’re going to be able to access your files because of SELinux rules. Even though people hate it, and given too that this is a local server, you may be tempted to shut it off and go about your business. This post encourages that you leave it on and simply reconfigure it to work with your server.

Basically, you’ll need to change the SELinux type on the entire directory structure starting with the root /opt/repos to public_content_t.

chcon -R -t public_content_t /opt/repos

Testing and Exposure

Run a few simple tests before giving the green flag for clients to start consuming your packages. What I’ll usually do is run a port scan with nmap to verify that the FTP port is open, which effectively means that vsftpd is running and listening on that port. Next, I’ll open a browser and navigate to the address of the server with the FTP protocol to see if I can view the contents. You can also test this in two other ways:

  • Install a FTP client on the repository server and attempt to establish a FTP connection to localhost (I don’t recommend this for probably a foolish reason).
  • Use a FTP client on a remote client and attempt to establish a FTP connection to the repository server.

Two really common issues at this point are either access or visibility to the repository contents. If access is an issue, make sure that vsftpd is running on your server and that firewalld is configured to permit ingress/egress traffic for FTP, especially if you’re using a non-standard port for FTP (egress traffic should be open by default, but these require direct rules through firewall-cmd to filter since the principal route of focus for firewalld is ingress). If you’re still having issues, make sure that your clients are configured correctly too (firewall, routing, etc.). Visibility of content can usually be traced back to one of three matters: a bind mount that wasn’t added to fstab (this would cause the mount to be dismounted on restart), incorrect DAC (755 should be sufficient; FTP wants the execute bit set), or incorrect SELinux type. The only other major rub is that if your storage strategy defines that your files are located on a NAS or other file server, you need to ensure that those mounts are established at system start (i.e. remote SMB share is mounted correctly).

To add the repository to a client, you can either package the public data into a RPM and have clients install it (much the same way that RPM Fusion does theirs), or they can manually add it to the listing under /etc/yum.repos.d. The file would look similar to this:

[reponame]
name=Custom Repo
baseurl=ftp://name-or-ip-of-ftp-server/centos
gpgcheck=0

Helpful Links

VSFTPD Online Manpages
Firewalld Homepage
SELinux FTPD
Configuring a Yum Repository File