- 5 Posts
- 31 Comments
1·12 days agoOk. Did a bunch more testing here tonight.
I have 2x 8tb SAS drives that have worked for several weeks now. Those show up in bios reliably regardless of which connector I use. So I think the HBA is working and the cables are good.
Of the 10x 10tb drives, 1 drive shows up reliably. It also seems to work on any connector I use. But it is the only drive the works.
Here is 8 of the 10 installed and only one showing in BIOS. I think the other 7 are not even spinning up.
An additional weird thing is that at least 1 other of the 10 drives did show up the first time I plugged it in from inside the xpenology VM. I did ‘hot swap’ that one in, but it then passed a SMART test. But since it passing the test and me pulling it, it hasn’t worked again. I’m also not positive which drive it is because originally I wasn’t expecting things to behave so weirdly, so I didn’t start taking notes…
As another test, I got a hold of a 9305-16i to see if the drives would read on that. And while that card would show up in BIOS, none of the drives – including the 8th that have always worked – showed up or spun up at all! I wonder if that card is not compatible with my mobo?
Is it possible other BIOS settings are interfering?
Ah. Will trying to hit swap damage the drives? Or just won’t work?
Drives are hot-swappable with this HBA, right?
Will give this a go! thanks!
One of the drives for reference:
Ok. Fair enough.
I’m still working on the first 2, but on the 3rd, yes there are a couple drives that are up and working on the card. 2x 8tb drives (also HGST).
They are SAS drives, not sata. Mobo does have SATA, but not SAS.
I do!
Thanks for the link!
Yah. Trying to make an Uber server.
Bunch of drives for a nas vm
3060 for a bit of a gaming pc cam for things like moonlight.
Rx590 is for a hackintosh vm to make local backups of my iCloud stuff.
And a unix vm to manage docker containers.
Also plan to put Plex in a LXC (I think that’s the term) on Proxmox so it can hardware decode using the igpu.
Welp. Seems I’m an idiot. I’m very much a Unix noob so assumed it was something that Unix I didn’t understand rather than check the physical card again.
I was definitely shipped an nvidia card, not the amd I bid on! So case opened with eBay!
Sorry for the dumb question. Thanks for the great help.
Stay tuned for my next dumb question. :)
How do you do this? Idk what to even google, exactly.
So in the end, the intel 1g NIC just worked, so I gave up for now on trying to get the 2.5g Broadcom working instead.
I might try to link aggregate later and use the 2.5g Broadcom and circle back on this… but we’ll see.
I also got the second GPU installed and it shows up, too. But it’s an Rx590 and is showing as an RTX2070… so I’ll be making another post shortly!
Thanks for all the input!
I have no experience with dmesg and also don’t know how to scroll the history since I’m not on a terminal app (since I can’t get the NIC up).
Anything here helpful?
Ohhh. In that last line. I wasn’t even looking at that, I assumed the block above that was setting up the primary NIC…
I’ll see if changing that interface name does it…
It’s late. I’ll have to pull the card and re run tomorrow. But here’s with the GPU in:
It’s an i7-14700 and an ASRock z690 extreme. I’m actually hoping to put a second GPU in the last PCIe slot so I can let proxmox use the iGPU, pass the 3060 into a Unix moonlight gaming VM, and pass an RX590 into a hackintosh VM.
ASRock Z90 Extreme.
Has 2 built in NICs. The intel 1g and the Broadcom 2.5g.
Trying to use the Broadcom here, though my Ethernet is only 1g house wide, so I could try the intel if that seems like it could help…
Ok. I’ll check it out.
Let’s say it is exhausted… what will get me more bandwidth? CPU or mobo…?
Only other pci-e card in at the moment is 16 line HBA seems to be basically 2 cards sandwiched on one board)…
This may also help, my HBA is there:
root@prox:~# for d in /sys/kernel/iommu_groups//devices/; do n=${d#/iommu_groups/}; n=${n%%/}; printf 'IOMMU group %s ’ “$n”; lspci -nns "${d##/}"; done IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] [8086:a780] (rev 04)
IOMMU group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Z690 Chipset LPC/eSPI Controller [8086:7a84] (rev 11)
IOMMU group 10 00:1f.3 Audio device [0403]: Intel Corporation Alder Lake-S HD Audio Controller [8086:7ad0] (rev 11)
IOMMU group 10 00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake-S PCH SMBus Controller [8086:7aa3] (rev 11)
IOMMU group 10 00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH SPI Controller [8086:7aa4] (rev 11)
IOMMU group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (17) I219-V [8086:1a1d] (rev 11)
IOMMU group 11 01:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Western Digital WD Black SN850X NVMe SSD [15b7:5030] (rev 01)
IOMMU group 12 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 13 03:00.0 Non-Volatile memory controller [0108]: Micron/Crucial Technology Device [c0a9:5415] (rev 01)
IOMMU group 14 04:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)
IOMMU group 15 05:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)
IOMMU group 16 05:08.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)
IOMMU group 17 05:09.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)
**IOMMU group 18 06:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)
IOMMU group 19 08:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)**
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:a740] (rev 01)
IOMMU group 2 00:14.0 USB controller [0c03]: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller [8086:7ae0] (rev 11)
IOMMU group 2 00:14.2 RAM memory [0500]: Intel Corporation Alder Lake-S PCH Shared SRAM [8086:7aa7] (rev 11)
IOMMU group 3 00:15.0 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0 [8086:7acc] (rev 11)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation Alder Lake-S PCH HECI Controller #1 [8086:7ae8] (rev 11)
IOMMU group 5 00:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11)
IOMMU group 6 00:1a.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #25 [8086:7ac8] (rev 11)
IOMMU group 7 00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #2 [8086:7ab9] (rev 11)
IOMMU group 8 00:1c.4 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 [8086:7abc] (rev 11)
IOMMU group 9 00:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #9 [8086:7ab0] (rev 11)
sorry this took so long… you know, life. trying that command altogether, I get this response: -bash: acpidump: command not found
trying just egrep “DMAR|IVRS” (in case they are two commands) seems to hang the terminal session.
I tried following a guide to enable PICe passthrough and get this. One important thing, there is no discrete GPU at the moment, I’m trying to pass through an HBA…
root@prox:~# dmesg | grep -e IOMMU [ 0.100411] DMAR: IOMMU enabled [ 0.254862] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1 [ 0.629143] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics [ 0.713978] DMAR: IOMMU feature fl1gp_support inconsistent [ 0.713979] DMAR: IOMMU feature pgsel_inv inconsistent [ 0.713980] DMAR: IOMMU feature nwfs inconsistent [ 0.713981] DMAR: IOMMU feature dit inconsistent [ 0.713982] DMAR: IOMMU feature sc_support inconsistent [ 0.713983] DMAR: IOMMU feature dev_iotlb_support inconsistent
I’m not a lawyer, but if I wasn’t planning to do anything related to the other ‘dedsec’, I wouldn’t even consider what the owner of that dedsec would think.
Businesses have the same name ALL THE TIME.
Unless you’re trying to piggyback on or undermine that other dedsec, I, the non-lawyer, can’t imagine how they’d have any standing to raise a concern.
Whatever you’re doing must have led you to the dedsec name (I assume it’s a website for dog-eat-dog security), there’s just no way in my (non-lawyer) mind that EA or Ubisoft has any right to stop you from saving dogs from being eaten.
I (the non-lawyer) would 100% not pay a lawyer for their opinion on this.