For example, does a 13B parameter model at 2_K quantiation perform worse than a 7B parameter model at 8bit or 16bit?

  • noneabove1182M
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    ahh makes sense, i just made a post and deleted the comment i made on it but it glitched and deleted twice so now my post has -1 comments lmao