Welcome back everyone!
This is part 2 in our 10Gb Home Network Series, and in this video we are going to be focusing
on peer-to-peer networking.
If you have not seen the first video in this series, make sure you stop here and check
out the introduction video since it covers some important concepts with 10Gb networking.
Alright!
Now that you have seen the first video, bragged to all your friends, and considered the possibilities
for your excessively fast home network, it's time to have some fun.
The main point of this video is to have some fun and keep things simple.
That means a minimal shopping list for our trip to the virtual store.
Check the video description for quick links.
All you need is two computers, two 10Gb network cards, and one cable.
That's it!
No ridiculously expensive 10Gb switch or magic required to participate.
Man, this kind of fun without an expensive switch should be illegal.
Okay, time for specifics with our shopping list.
And to that point, compatibility is a very important consideration if you are going to
do this the right way.
The two computers you select need to each have an open PCIe x8 (Gen1) slot.
You need a Gen2 PCIe slot if the 10Gb card has dual ports.
If you are uncertain, be sure to check the requirements of the card you purchase.
Next on the list is two 10Gb SFP+ Network Interface Cards (NIC).
Some popular brands include Intel, Mellanox, and Chelsio.
Each vendor has different feature sets on the various 10Gb NICs they offer.
Make sure you purchase the same two cards for maximum compatibility.
And don't forget to consider what operating system you are using with the 10Gb NIC.
Support for 10Gb is not as broad as the 1Gb market, so getting the right driver for your
operating system will be vital to maximize your performance.
For this video I'm going to be demonstrating the Mellanox ConnectX-2 Ethernet Adapter.
At the time of this recording I purchased two cards for $18 each on EBay.
After-market prices are sure to fluctuate, especially if everyone rushes to E-Bay for
these cards.
Brand new, 10Gb cards are very expensive, because vendors are targeting the enterprise
market.
Lastly you will need, at least, one Direct Attach Copper (DAC) cable, so you can directly
connect PC number 1 to PC number 2.
These cables come in three different lengths: 1m, 3m, and 5m.
That's 3ft, 10ft, and 16ft.
This is called a peer-to-peer connection.
This is a great way to keep things simple and cheap.
Make sure you check the video description, section titled Shopping List, for helpful
links to the items I just mentioned.
Now that you have your geek gear list, it's time to pull the trigger on some hardcore
upgrades for your network.
A few places I frequent are Amazon and EBay.
If you don't want to pay full price, look to EBay.
Before making any purchases, consider the following suggestions.
1) Make sure the card you purchase has a full height bracket.
A lot of the cards listed came from servers with half-height brackets. 2) Get a single
port card for reduced price.
Consider a dual port card if you plan on making multiple connections to the same system.
Example: PC #1 connects to your storage server and PC #2 connects to the same storage server.
This scenario would require three cards: 2 single port cards, and one dual port card.
3) Check your operating system Hardware Compatibility List (HCL) to verify the card you are about
to purchase works with your operating system.
I will list combinations I have tested in the video description for your convenience.
4) Get matching cards to maximize your compatibility. 5) Measure the distance between systems you
plan to connect, so you know how long the DAC cable needs to be.
The longer cables tend to have more bulk and less maneuverability.
Use the links in the video description to make your purchase and unbox when the gear
arrives.
Ahh yes, that will be a glorious day indeed!
Install day has arrived!
Crack your workstations open.
Install the 10Gb Ethernet adapter into an open PCIe x8 slot securely, and button things
back up.
Repeat this procedure for workstation #2.
Now connect up the two systems with the Direct Attach Copper (DAC) cable as shown.
If you insert the cable module facing the proper direction, you should feel the module
click into place.
If you need to remove the DAC module, you will need to push the cable towards the card
while you pull the release strap.
This should allow the module to be removed.
If the module does not freely slide out, do not force it out by pulling hard.
Try again as the video shows.
Now that both adapters are installed and the copper cable is connecting them together,
you officially have a 10Gb network.
At least physically speaking.
Now, celebrate early with your best Truffle Shuffle dance moves.
Next we are going to verify that your Ethernet adapter is not being throttled by the PCIe
interface.
10Gb Ethernet adapters are going to tax your system one way or another, so It's important
to rule out your PCIe bus speed as being a limiting factor.
On Windows run "msinfo32" (System Information), and search for the name of your Ethernet adapter.
In my case I would search for "Mellanox".
You can also try searching for "10G".
Once you find your Ethernet adapter, look at how it registers.
Mine displays as "Mellanox ConnectX - PCIe 2.0 5GT/s".
Or, using PowerShell run Get-NetAdapterHardwareInfo and check the PCIeLinkSpeed and width column.
Long story short an x8 PCIe 2.0 slot will give you 32Gbs.
That's plenty to handle a single (10Gbps) or dual (20Gbps) port card.
On the other hand an x8 PCIe 1.1 slot will only give you 16Gbps, so that is fine for
a single port.
But if you use a dual port card here, expect reduced bandwidth.
Many will attempt to use older systems, so I just want people to understand what the
limiting factors are.
If you are on a Linux system execute "dmesg | grep -i 10g".
The log may indicate you are running at reduced capacity if that is the case.
Now that your hardware is installed, it's time to load a driver and update the firmware
on the Ethernet adapter.
Start by downloading the appropriate driver for your card.
You may have to check the "legacy" section on the drivers page as you will be working
with older model Ethernet adapters.
Once you have the driver downloaded, start the install and walk through the installation
screens.
You will need to restart your computer at this point for the drivers to register with
your operating system.
Now that the drivers are installed, we need to verify the latest firmware is loaded on
the Ethernet adapter.
Often this step is overlooked, but important to avoid compatibility and performance problems.
Download the firmware update tool.
In my case the Mellanox utility is called "mlxup".
I copied it to the Windows slash System32 directory so it would be in the system path.
Run the update utility to query your card and see what version firmware is already installed.
Next you download the matching firmware if you are not running the latest version.
The process will be different for Intel and Chelsio cards.
Once you have the firmware downloaded and extracted, you can run the update utility
to flash the Ethernet adapter with the latest version.
In my case it was already up to date.
Repeat this procedure for the other computer.
It's finally time to configure our new adapters.
The idea is configure your 10Gb Ethernet adapters on a totally different network.
This will avoid network path issues and simplify the network configuration.
Use the displayed configuration slide to set your 10Gb adapter settings.
Continue, when you are finished, with the network and advanced property settings.
Next open File Explorer and browse to c:\windows\system32\drivers\etc and edit the "hosts" file.
You will need administrative privileges to edit the file.
You will add two entries making it easier to reference systems on your high speed network
segment.
The hosts file is a local version of a DNS entry.
In this segment we will be setting up tools to aid in the testing and use of your new
10Gb network.
The first tool is called iPerf.
It is a command line utility, but there is a visual front end you can add called jPerf.
This simplifies the tool and makes it a little easier to use.
The second tool is called SoftPerfect RAM Disk.
This allows you to create a RAM disk on each system and transfer using high speed RAM.
This is a good method for ruling out your hard disk drive as a performance bottle-neck.
Start by installing jPerf, which should include the iPerf binary (linked in the video description).
Run a jPerf server instance on PC #2 and start jPerf client on PC #1.
Configure client settings as displayed.
Or you can just use the command line if you prefer.
Match the settings I use, and execute a 30 second job to get a sense of how your system
is performing.
Bring up Task Manager and monitor your compute and network utilization to verify nothing
odd is happening and things are functioning as you expect.
Everyone's speeds will vary for the following reasons: processor speed, processor degree
of parallelism, RAM amount, bus speeds, hard drive speed, protocol, and of course network
bandwidth.
Your limiting factor could be on the send or receive side of the equation too.
Where iPerf is great for testing network throughput, SoftPerfect RAM Disk is great for file transfers
without being inhibited by potentially slow disk drives.
Download and install the RAM Disk software on both workstations.
Configure RAM Disk as shown in the video and adjust to your available RAM.
Next, find a large file that will fit in the newly created RAM Drive, share your RAM Drive
on workstation #2, and connect to it using the hosts file alias we created earlier.
Using this method we are connecting workstation #1 to workstation #2 using the 10Gb network.
If you just use the normal host name, Windows will likely connect to the remote system using
your standard 1Gb connection, so be aware.
Now that you've seen what can be done with 10Gb Ethernet speeds, I'm sure you are wondering
how else you can take advantage of this tech.
I know I was!
So I started investigating my options for expanding 10Gb Ethernet to the remaining systems
in my Home Area Network (HAN).
I quickly realized 10Gb switches were outrageously priced.
So I did what any geek confronted with a technical challenge would do.
I built my own switch!
Check out my third video, in the 10Gb Home Network Series, where I teach you how to build
your own switch and bypass the ridiculous pricing for 10Gb commodity switches.
Don't forget to subscribe, like and comment to join the discussion.