A backdoor would still be highly auditable in a number of ways even if inspecting the weights isn't viable.

There's no possibility for obfuscation or remote execution like other attack vectors