AMP training with RTX4090 incredibly slow

[giu3232Participant]

Hi guys,
I managed to make the AMP training works.
Thing is it’s incredibly slow. I let it run overnight and it only got 15.000 iterations using my rxt4090.
It seems wrong.
Any idea if it s normal and if not what happens?
=================== Model Summary ====================
== ==
== Model name: WAYNE KNIGHT 1024 AMP_AMP ==
== ==
== Current iteration: 219 ==
== ==
==—————– Model Options ——————==
== ==
== resolution: 640 ==
== face_type: head ==
== models_opt_on_gpu: True ==
== ae_dims: 300 ==
== inter_dims: 512 ==
== e_dims: 80 ==
== d_dims: 80 ==
== d_mask_dims: 30 ==
== morph_factor: 0.5 ==
== uniform_yaw: False ==
== blur_out_mask: False ==
== lr_dropout: y ==
== random_warp: True ==
== ct_mode: none ==
== clipgrad: False ==
== autobackup_hour: 1 ==
== write_preview_history: False ==
== target_iter: 0 ==
== random_src_flip: False ==
== random_dst_flip: False ==
== batch_size: 4 ==
== gan_power: 0.0 ==
== gan_patch_size: 640 ==
== gan_dims: 512 ==
== ==
==——————- Running On ——————-==
== ==
== Device index: 0 ==
== Name: NVIDIA GeForce RTX 4090 ==
== VRAM: 20.85GB ==
== ==
======================================================
Starting. Press “Enter” to stop training and save model.
Thanks.
J.

[Usertest21Participant]

Hey did you find any solution? and what version of DFL are you using?

[Author]

Posts