Download PDF

On-net servers.

Many ISPs want to install test servers inside their network (hence "on-net") to allow them to segregate on-net and off-net performance. SamKnows provides a process to install, update, and monitor the SamKnows applications on these test servers.

Why use on-net?

The overwhelming majority of test servers used by SamKnows customers are off-net, i.e. hosted on the public internet in some fashion. We believe that reporting results to targets off an ISP's own network represents a "real world" experience for end users. However, we do recommend that ISPs install and test against on-net servers as well as off-net.

With both on-net and off-net servers in use, customers can see the difference between the performance of the ISP's own network and that of the public internet. The results can be used to troubleshoot peering links, routing issues, or simply rule out any capacity problems within the ISP's own network.

Requirements

On-net test servers can be either virtual machines or dedicated hardware. For dedicated servers, we strongly recommend that they only operate as test servers and are not used for any unrelated purpose (for example, as a web server or file server).

The minimum specification of a test server is as follows:

  • CPU: Quad Core Xeon (2GHz+)
  • RAM: 8GB
  • Disk: 250GB
  • Operation System: CentOS 7.x 64-bit or Redhat Enterprise Linux 7 64-bit
  • Connectivity: Gigabit Ethernet connectivity, with 1Gbps upstream (10Gbps preferred)
  • IPv4 connectivity

Measurement servers must be able to sustain 1Gbps throughput. Should your project involve testing connections of 500Mbps or faster, we strongly recommend provisioning 10Gbps servers to ensure there is no network contention. Additionally, if you wish to use Redhat Enterprise Linux, you must provide the appropriate Redhat License details for each server.

At a minimum, one publicly routable IPv4 address must be provisioned per server. The test server must not be presented with a NAT'd address. It is preferable for any new test servers to also be provisioned with an IPv6 address at installation time.

DNS records must be configured for each server before installation of SamKnows applications can proceed. We recommend using separate DNS records for IPv4 and IPv6, for example servername-v4.company.com and servername-v6.company.com to make it clear which protocol is being used at any time.

Server Management

SamKnows uses the popular agent-based management system "Puppet" to control and manage our server infrastructure. More details on Puppet can be found in the Methodology section of this documentation.

After the operating system has been installed by the customer, all documentation has been completed, and the installation instructions have been followed, then Puppet will download and install the SamKnows applications on the server. Puppet will also ensure that the server is kept updated with the latest versions of the SamKnows applications, as well as ensuring that these applications are in a valid running state.

As part of the default on-net server installation, SamKnows will configure monitoring using our Nagios monitoring system. This involves a combination of active tests to the server, for example to ensure that services are listening on the correct port, and passive tests where the server checks its own process status and sends this information back to our monitoring system. We recommend that customers continue to run their own monitoring of the basics of connectivity, power, and network capacity.

The customer is responsible for ensuring the health of the operating system, the server hardware, and the local network.

Provisioning on-net test servers

ISPs are requested to complete an information form for each test server they wish to provision on their network. This will be provided by your SamKnows account manager. This will be used by SamKnows to configure the test server on the management system.

Additionally, if you have any special requirements - for example, the creation of an administration account on the server for your own use, or particular firewall rules that must be implement on the server - then please provide us with this information before installation.

Installation proceeds as follows:

  1. Ensure that your test servers meet the minimum specifications.
  2. Ensure your servers have the necessary firewall rules permitted.
  3. Complete and return the test node installation form.
  4. Await confirmation from your SamKnows account manager that the test servers have been configured on the SamKnows back-end.
  5. Ensure that the results of the following commands are all correct:
Check commandExpected resultPotential problemif not correct
hostnameservernamePuppet SSL certificate DN will not match
hostname -fservername.company.comPuppet SSL certificate DN will not match
grep search /etc/resolv.confsearch
company.com
DNS/Reverse DNS lookups may fail
(echo > /dev/tcp/pm.samknows.com/8140) > /dev/null2>&1 && echo "OK"OKUnable to contact the SamKnows Puppet master server(s)

If the output for any of the above check commands do not concur with the expected result, please do not proceed until the issue has been resolved.

  1. If the output for all of the above check commands is correct, you may proceed with the following commands to install & configure Puppet, which will then install the SamKnows applications and configure the server:

    yum -y update && rpm -Uvh https://yum.puppetlabs.com/puppet5/puppet5-release-el-7.noarch.rpm

    yum -y install puppet-agent

    source /etc/profile

    echo -e "[main]\nserver = pm.samknows.com" >> /etc/puppetlabs/puppet/puppet.conf

    puppet agent --waitforcert 15 --test

If this process fails to complete successfully, it likely means that your server has not been provisioned on our systems. Please ensure you've followed the steps above, and if so, contact your SamKnows account manager with details of the error message.

Firewalling of on-net test servers

It is preferred that the test servers do not sit behind a hardware firewall or a network Access Control List as firewalling is usually managed on the testserver. If a firewall is used, then care must be taken to ensure it can sustain the throughput required above. Additionally, the following rules must be permitted at a minimum:

Inbound firewall rules required

Note: We recommend opening a range of TCP and UDP ports, 5000-7000, for proper operation of our services. We list the primary ports used by our applications for informative purposes.

Source IPProtocolPortPurpose
77.89.189.17/32TCP & UDPALLRemote management from the office (Main IP)
89.105.103.193/32TCP & UDPALLRemote management from the office (Backup IP)
2a00:f18:33::/48TCP & UDPALLRemote management from the office (IPv6)
83.142.229.43TCP22Remote management from Bastion Server
ALLTCP80 & 443Test HTTP(S) Traffic (nginx)
ALLTCP & UDP5000-7000Test Traffic (SamKnows Applications)
ALLTCP8080Test Traffic (skhttp_server)
ALLTCP8443Test Traffic (skhttp_server)
ALLUDP8444Test Traffic (skhttp_server)
37.220.21.130TCP5666Nagios NRPE Active Monitoring Traffic (lnman1.samknows.com)
2a02:2658:101d::2TCP5666Nagios NRPE Active Monitoring Traffic (lnman1.samknows.com)

Outbound Firewall Rules

Note: These are only required if outbound access is denied by default.

Destination IPProtocolPortPurpose
159.69.58.62TCP8140Puppet Management Traffic (eupm1.samknows.com)
2a01:4f8:231:981::2TCP8140Puppet Management Traffic (eupm1.samknows.com)
23.92.20.243TCP8140Puppet Management Traffic (newpm1.samknows.com)
2600:3c03::f03c:91ff:fe24:b483TCP8140Puppet Management Traffic (newpm1.samknows.com)
139.162.246.10TCP8140Puppet Management Traffic (lonpm1.samknows.com)
2a01:7e00::f03c:91ff:fe24:f8e4TCP8140Puppet Management Traffic (lonpm1.samknows.com)
213.52.128.57TCP389LDAP Data Access (ldap1.samknows.com)
2a01:7e00::f03c:91ff:fe1f:3abTCP389LDAP Data Access (ldap1.samknows.com)
172.104.131.94TCP389LDAP Data Access (ldap2.samknows.com)
2a01:7e01::f03c:91ff:fefb:de27TCP389LDAP Data Access (ldap2.samknows.com)
172.104.139.223TCP80 & 443SamKnows managed repository mirrors (mirror.samknows.com)
2a01:7e01::f03c:91ff:fe80:ea3eTCP80 & 443SamKnows managed repository mirrors (mirror.samknows.com)
23.92.16.59TCP80 & 443SamKnows managed repository mirrors (mirror.samknows.com)
2600:3c03::f03c:91ff:feb6:8902TCP80 & 443SamKnows managed repository mirrors (mirror.samknows.com)
139.162.9.228TCP80 & 443SamKnows managed repository mirrors (mirror.samknows.com)
2400:8901::f03c:91ff:feb6:26e6TCP80 & 443SamKnows managed repository mirrors (mirror.samknows.com)
8.8.8.8UDP53Google DNS
2001:4860:4860::8888UDP53Google DNS
8.8.4.4UDP53Google DNS
2001:4860:4860::8844UDP53Google DNS
139.162.167.123UDP123NTP Service (ntp.samknows.com)
2a01:7e01::f03c:91ff:fe2a:4c03UDP123NTP Service (ntp.samknows.com)
151.236.222.193UDP123NTP Service (ntp.samknows.com)
2a01:7e00::f03c:91ff:fe78:b004UDP123NTP Service (ntp.samknows.com)
66.228.32.104UDP123NTP Service (ntp.samknows.com)
2600:3c03::f03c:91ff:fe2a:ad71UDP123NTP Service (ntp.samknows.com)
139.162.33.192UDP123NTP Service (ntp.samknows.com)
2400:8901::f03c:91ff:fe2a:4c8aUDP123NTP Service (ntp.samknows.com)
37.220.21.130TCP42217Netstat RRD Traffic Monitoring (lnman1.samknows.com)
2a02:2658:101d::2TCP42217Netstat RRD Traffic Monitoring (lnman1.samknows.com)
37.220.21.130TCP5667Nagios NSCA Passive Host Monitoring (lnman1.samknows.com)
2a02:2658:101d::2TCP5667Nagios NSCA Passive Host Monitoring (lnman1.samknows.com)