3 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
FeepingCreatureandGitHub	260a5ca5d9	Allow the prompt request to specify the prompt ID. (#8189) ... This makes it easier to write asynchronous clients that submit requests, because they can store the task immediately. Duplicate prompt IDs are rejected by the job queue.	2025-07-14 14:48:31 -04:00
FeepingCreatureandGitHub	9c98c6358b	Tolerate missing @torch.library.custom_op (#7234) ... This can happen on Pytorch versions older than 2.4.	2025-03-14 09:51:26 -04:00
FeepingCreatureandGitHub	7aceb9f91c	Add --use-flash-attention flag. (#7223) ... * Add --use-flash-attention flag. This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.	2025-03-14 03:22:41 -04:00