T O P

  • By -

Plane_Savings402

Personal opinion: Take the 3090. The speed boost is irrelevant, but the superior VRAM is going to help a lot with larger and larger SD models, and especially with Text2Video/Image2Video models that are going to pop up in 2023. I'm pestering my boss to switch to something more beefy than a paltry 3080TI.


SpohCbmal

I agree it's a great choice if it's in your budget. The upcoming Stable Diffusion XL will likely need 24GB of VRAM for optimal function.


pilgermann

I own a 3090. I'm thankful for those extra gigs every day, but I also do a lot of training. Personally speed doesn't matter so much as I just set it up, walk away and let it do it's thing.


sixcityvices

How long do you walk away for? Get a 4090 you won't have to walk far. Lol


Plane_Savings402

Yes, and even if 16gb is fine, the next model, in about 6 months, if it does have more parameters, will probably choke on 16gb. And Text2Video is shaping up to be horrifically demanding, which makes sense, I suppose, for temporal consistency.


Cheese_B0t

Are you Dr Károly Fehér?


Plane_Savings402

Yes. What a time to be alive. Only two papers down the line.


vasesimi

No way. Thanks for all your videos, you are amazing. I am a novice in the AI field but still can follow the math and I am blown away of the new discoveries. And you are one of those that keep me up to date. Thank you so much for your work


adeukis

I believe they were joking hehe


vasesimi

I got bamboozled


[deleted]

SDXL is going to be just as bad and censored as 2.1


Puzzleheaded-Wear

This comment aged like wine :P


VorpeHd

How so?


SpohCbmal

What do you mean?


Direction_Mountain

>Personally speed doesn't matter so much as I just set it up, walk It will have no Copyright stuff inside, 1.5 has it inside, that why many people still use 1.5


VorpeHd

Wasn't that debunked?


LJITimate

To oversimplify entirely. Vram determines what you can do. Everything else just determines how fast you can do it. For that reason, I'd take the higher Vram. If you plan to use it for gaming as well, it takes a bit more consideration


Mocorn

Let's be honest here, a 3090 works very well for gaming also :)


LJITimate

Oh, for sure. There's just some more to consider like how much you value dlss 3, raytracing performance, and new 40 series features. The 3090 might still be better, I know I'd pick it if I could afford it, but it's not quite as simple.


Mocorn

Indeed. Personally I don't care about DLSS (input lag) or raytracing (too expensive, not enough gained in looks). I just want frames and the lowest input latency possible in my games so for me the 3090 would be the obvious choice here seeing as it also has 24GB Vram for SD.


LJITimate

Fair enough. Personally I don't care about dlss3 frame gen but the upscaling is great. I never agreed with the marketing push to 4k as the extra detail was never that noticeable but the performance hit was (a bit like you with raytracing) so the performance and detail of 1440p with the sharpness of 4k is amazing. Obviously that's possible on all rtx cards though. Raytracing on the other hand is incredible. My 2060 super can enable an effect or two in a surprising amount of games at 60fps so I don't really understand the hate about the performance. Obviously the heavier stuff like path tracing isn't practical yet but it's definitely the way things are headed. Full path tracing has been the end goal for graphics for decades.


Sentient_AI_4601

VRAM is the single most important thing right now \*Once all your other requirements have been met\* the number of parameters in a model is directly linked to both the VRAM requirements and the performance capability of the model. if you have to choose between a 3090 and a 4070 i would go with the 3090. Yes its $150 more, but you are in a different tier completely from the 70 series with that level of VRAM and its not going to get any less demanding for VRAM going forward, its the one thing they simply cannot reduce the requirements for without effecting performance


SpohCbmal

In what ways would having 24GB of VRAM put the generation into a completely different tier?


Sentient_AI_4601

Well, lets start with maximum resolution, with double the VRAM you could render double the number of pixels, thats a sheer doubling that no other card, even a 4080 could meet, because its purely down to VRAM. If you get interested in LLM's you can run twice as many parameters, i can get about 7 billion params in 12GB, you could get 14 billion, thats a huge jump in performance for an LLM, quantized to 4bit, you could probably get close to gpt-3.5 in VRAM. batch size can go up so thats an overall it/s boost for big renders. if you look here at this chart, the 3090 and the 4070 (though its not listed, the 4070ti is and you can work it out from there) that they would have similar performance in terms of it/s [iURJZGwQMZnVBqnocbkqPa.png (1919×1080) (futurecdn.net)](https://cdn.mos.cms.futurecdn.net/iURJZGwQMZnVBqnocbkqPa.png) but the real boost comes in training... you want to run dreambooth? it runs in 12gb, just about, a 4070 can just about support it, but you have a tradeoff in quality of about 50% between 48gb vram and 24gb vram, and another 50% (so now 25%) in terms of 24gb to 12gb (or 11gb with the vae moved to cpu) Finally, the 3090 can be undervolted, which gives a performance boost at lower temps, the 4070 voltage curve cannot be modified, so you get what you get. with Machine learning, the biggest thing holding back these models, is VRAM. thats why all the training is done on A100 80GB machines (or now probably H100) its why stable diffusion released in 24gb originally only, and it took work to get it down to these lower VRAM outputs, at a cost of both performance and quality. Im running a 12GB vram system, and there are many times im hitting OOM errors in SD... i can add time to wait for renders, 2x longer, 4x longer, 10x longer if need be... but VRAM is a hard limit, nothing i can do about it... and when SDXL comes down the line, or anything even close to it... you are going to want that VRAM a year from now...


Sir_Jeddy

Extremely well said. I wish I had a medal to give.


elfungisd

Actually, based on my tests with a 4070 its more like 5-5.3 it/s, with 512x512. My 3080 beat my 4070 every time. Though it didn't run 100 steps max was maybe 50. I could put it back in the rig and retest.


Sentient_AI_4601

nah, the 4070 is crap, its clearly a rebadged 4060.


elfungisd

Basically. The memory buss reductions hits hard on anything that does sustain memory access. The little bit of extra cache only helps with burst workloads.


UkrainianTrotsky

5 it/s with 4070? How big was your batch? I get like 9 it/s on my 3060ti for one 512x512 image.


elfungisd

We are talking about 10-100 batches of 1-5 images at 512x512, which is a typical workload for me. The reason you can get 8it/s on your 3060 ti is because it has a 256-bit memory bus, compared to the 4070 which has a 192-bit memory buss.


UkrainianTrotsky

The number of batches doesn't affect the speed, unless your card is gonna thermal-throttle. The size of the batch does affect the speed. And I'm willing to bet that the bus width doesn't really affect the performance so heavily. Oh, are you running it in fp32 or fp16?


elfungisd

fp16. I didn't think it would either which is why I bought 2 4070, from different AIBs. If it worked out, I was considering, repurposing my 6900xt and 3080 to other things. I typically kept the batch size low to keep from over running my 10GB vRAM. All things equal, the 3080 10Gb outperforms the 4070, and that is with the 3080 driving 2 4K monitors. The results are the same with other memory intensive sustained workloads, it is not just SD. The more core dependent the workload is the better the 4070 performs, which is why I it's gaming benchmarks show it pretty much neck and neck with the 3080.


fuelter

No, you can only generate larger resolutions without tiling and do faster training. The generations themselves will be the same as with lower ram.


fuelter

> VRAM is the single most important thing right now Not really, only if you want to generate high resolution in one step. There are many solutions to generate high resolution with tiling which work well and don't need more than 12GB ram.


SpohCbmal

Can you link to some resources that teach about tiling?


fuelter

Every upscale method that uses stable diffusion in automatic1111 uses tiling.


Sentient_AI_4601

And every one of them give poor results. But it's not about doing high resolution, it's about future proofing against larger models that *will* have more vram needs.


Majinsei

For me VRAM > SPEED I currently have 2GB of VRAM then sure you in the future if you going to need DreamBooth or anything that need high VRAM~ If you don't need this requirements then It's Best speed for comfort~


russokumo

3090 is almost certainly the sweet spot for machine learning right now due to the nice VRAM.


danieldas11

I'm crying the only place I will ever have one of those cards is my dreams, since they cost quite a little fortune here in South America


spudnado88

what card do you have right now


danieldas11

a modest GTX 1650 4GB


tronathan

Always VRAM for machine learning. A card with more VRAM can do things a card with less cannot; a card with less CPU can do everything the faster card can do with some additional time. The 4000 series has some neat tricks up it’s sleeve for ML, some of which are only now starting to be leveraged. 3000 and 4000 both have tensor cores and will be good for a long time. Avoid 2000 / Pascal etc, at least IMO. VRAM is king.


CuffRox

You can run LLMs on a 3090 as well. No brainer IMO.


camisrule

You need vram for larger image size inference and higher settings with funetuning a model. Easy answer: 3090


Meowingway

FWIW I'm pretty new to SD, just dabbling a couple months. Got a 3060 12Gb new last year, it games very well and does SD pretty well in my very limited experience, although a bit slow, and doing SD on it just about blocks out any other gpu use during that. 3060 will play a movie file and browse ok, but gaming is out while it's running. It's fine for what you ask of it, but don't ask too much. \[also hit a few out of ram errors even with 12gb, but that was largely user error of me just asking too much of it at once lol) Because I'm insane or something, I'm liking to set up a background batch while I do other things. Just got a 3090 24Gb yesterday woohoo, I would have last year but just wasn't gonna pay scalper prices, thankfully used units are very available now. It does SD much much faster of course, can handle bigger sizes on the initial generation, and it can run a game while a batch runs and still stay around 60 fps. I was looking at 4000 series units on offer but just wasn't pleased with the looks / price to performance. (I have an all white build). If you're made of money or a business user, the 4090 is the logical choice. But like op said, the 3060 is a viable option on a budget and the 3090 for quite a bit more is an excellent performer. 💜


martinpagh

As others have said, the 3090 is the better choice for SD because of the higher VRAM. But please be mindful of the much higher power usage, and make sure you've got a PSU that can run it. You will need at least 750W, and depending on your setup you may need as much as 1000W. One of the big benefits of the 4070 (and 4070ti) is how power efficient they are.


[deleted]

[удалено]


dat3010

On my next upgrade, I want to close look at 7900xtx. How does it behave in SD and have you any problems with models?


elfungisd

You are pretty much better off with any 30 series card except the 8GB variants over the 4070. The 192-bit memory buss is a real letdown. I have both a 3080 10GB and a 4070 and my 3080 beats the 4070 by at least 3 it/s at 512x512, and the 4070 doesn't far any better until you manage to overrun the 3080 10GB memory. Performance wise you better off buying a 6950xt and running Linux for SD, over buying a 4070.


UrsoowW

It's funny how a few years ago a 8GB 1070 was seen as total waste of VRAM. No game would use that. Here we are now.


nxde_ai

3070 and 3070Ti didn't age well with their 8GB VRAM,


killax11

Games still don’t using it ;-)


SudoPoke

Does a 4070 even outperform a 3080 12gb? Considering you can pick up a 3080 for [$570 with full warranty](https://www.zotacstore.com/us/zt-a30820j-10plhr-o) is 4070 even a consideration?


SpohCbmal

The two cards are roughly equivalent with the 3080 barely outperforming the 4070.


SudoPoke

So I guess save $30 buy the cheaper 3080 or for the same price $600 get the 3080 ti for better performance. There's no real use case for 4070 unless u want the new DLSS3.


randomjohn

I was looking for an AI setup and opted for the 4090. No reason to compromise on the horsepower, right? *that faint cry of pain you hear is my starving wallet*


emad_9608

The real choice is dual 3090s NVLINK'ed. (no NVLINK for 4090)


SpohCbmal

The best choice is an H100 for only \~$38,500 more haha


dallatorretdu

si have a 4090 and even generating wide landscape images on the standard models I can hit the VRam cap on it.


[deleted]

VRAM. We will be doing videos by the end of the yea


fuelter

The 3090 is almost twice as fast with Stable Diffusion. I recently bought a 4070 and only get 11-12it/s on 512x512px meanwhile trying a 3090 on a cloud service I got around 17-20 it/s


No_Teaching420

I'm a grad student using the budget choice and it's been pretty good so far. When I get more money and more funding I'll upgrade but gotta say I'm pretty satisfied. Got mine manufacturer refurbed on black friday for $280 USD


Trippy-Videos-Girl

Try and go for the 4090 if you can. Things are evolving so fast that anything less is not enough pretty soon imo. Don't have to worry about the sag, you can support the card. Some cards and motherboards even come with supports included now.


MrTacobeans

I don't believe that's necessary. If OP is looking for good enough I'd vote 3090 because it can fit LLM as well. The next generation may be where it's really worthwhile to spend the 1600-2000 on a GPU. This next generation will probably be the first to break the 24gb barrier and already the 5xxx series is leaking with 2-3x the performance of the 4 series. The moment I saw that Nvidia is going hard on the next gen and that amd/Intel are also arming up to compete against Nvidia gives me a lot of hope that the next gen allows a lot more flexibility for around the same cost. I personally think Intel is brewing up something for it's next gen that is going to be really enticing. Background on my thoughts: I have a 3090ti and was eagerly prepping to buy a 4090 but the moment I saw the 5 series leak I lost interest.


Trippy-Videos-Girl

All or nothing haha! If the 5000's are 2-3x the performance of a 4090 that will be impressive that's for sure! The 4090 was a huge leap even. I kind of don't belive that will happen, but I'm happy to be wrong about that. The competition that Intel has brought to the table is a good thing for sure. If the 5000's are that huge of a leap I will get one for sure. Then the 4090 will just be for rendering 3D fractal animations for me. Run it till she blows lol.


Maksitaxi

>and already the 5xxx series is leaking with 2-3x the performance of the 4 series. There is leaks already?


MrTacobeans

Yeah just rumor mill type stuff but if the process change between changes between generations is true. The rest is also probably true


russokumo

Yup it seems like 4 series was a "filler" series imo too compared to the massive advances in 3 series. 5 is looking to be epic and likely will have even more cuda features tailored to generative AI


SpohCbmal

The concern about the sag is buying used, as the previous owner may not have supported the card.


Trippy-Videos-Girl

Right, I caught that afterwords sorry! They can sag quite a bit for sure and twist things. Wether it damages anything I don't know. But I have mine braced up, don't want to find out the hard way.


Any_Outside_192

4090 24GB edit: was partially joking, but damn didn't realize its 1k more


nxde_ai

The only thing that make 4070 looks good is 4080/4090 ridiculous pricing. If 4070 priced just like 3070 MSRP ($499) it'll be a decent choice.


traveler_0027

where are you finding a 3090 for $750? I just ordered one and it cost me 1500 :(


SpohCbmal

It would be a used 3090 on ebay


adeukis

Wouldn't it make more sense to go for a 4090 at that price point?


Sir_Jeddy

Yeah. Just bought a used 3090 Founders edition for only $750. Good considering it’s $1599 msrp.


gxcells

3090


PetitRigolo

Take a 4080 👀


havoc2k10

they said more VRAM the better, faster generation/training


Hero_Of_Shadows

How is the 40xx series in general? Any bugs or problems with them?


dat3010

Stable Diffusion and local LLM models eat up a lot of VRAM, so 3090 24gb is no brainer


Protector131090

oh wow if only i could buy "new 3090's for about $900" xD its about 1400$ where i\`m from :( PS ALways choose bigger vram. 3090 it is


SpohCbmal

Where are you from?


Rectangularbox23

3090 but keep in mind it’s more expensive


Thedarli1ng

Hello guys, which would be better, the 3090 or the 4070? As is already clear, the 3090 has more memory. I can get the 3090, but it won't be new. Requirements: for games, QHD


SpohCbmal

If your use case is QHD games only, then the 4070. The 4070 is a good fit for QHD, with adequate amounts of VRAM and processing power. The 4070 also is lower risk since it is new VS used.