This is a simple fix but not a simple problem and it still doesn't make sense to me.
But in a nutshell if your target proxy server works fast when accessing directly over SSL then this may be your issue.
It seems SSL does not play nicely when the target proxy destination/host has a riduculously long key (such as 8192 bits long). Now this is normally not a problem, in fact the target server could be accessed with hardly any delay directly despite such a long key.
However when throwing a Proxy and Reverse Proxy with SSL in front of it, made requests take 20-30 seconds (not milliseconds but actual seconds).
I found many proposed solutions or people saying it should be that slow, but that's just not the case, yes SSL is slower but it shouldn't be this slow. Disabling or enabling all SSL protocols also made no difference.
Solution
The problem was instantly solved by replacing the target server's SSL key with a 2048 bit one ( a smaller one). I think this is a big where the SSLProxyPass just can't handle the request properly when the target SSL server has a large public key.
Additional Performance Improvements
Use the "CacheEnable disk /" directive in httpd.conf
apache, ssl, proxy, solutionthis, doesn, nutshell, server, accessing, destination, riduculously, bits, accessed, requests, milliseconds, solutions, slower, shouldn, disabling, enabling, protocols, instantly, replacing, sslproxypass, additional, improvements, quot, cacheenable, disk, directive, httpd, conf,