performance with large datasets

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #880
    joncas
    Participant

    The current master-2019-2 implementation does not seem to scale well to large datasets.
    Running a test with 4000 customers, 7000 products and 1500 invoices, it takes 42 seconds to load “manage invoices”.
    Investigating with the XDebug profiler showed that half the time was spent formatting all the currency values. Replacing that with displaying the raw amounts halved the time, but loading all invoices ahead of time rather than just the first 25 is still not workable.
    I had tested the 2013 version for a friend, which loaded only the initially visible number of records and fetched the rest on demand, which works fine with any number of invoices.
    For creating and editing the invoices, the problem is worse, because all the customers and all the products get loaded ahead of time, so opening a new invoice takes 46 seconds and runs 88000 sql calls.
    In the 2013 version I had managed to avoid having all the products loaded by replacing the product SELECT with a text input where the user enters the product id.
    When I tried that with the current version, loading all the customers still causes too much of an overhead, so something in the database access seems to have a much larger overhead than before, but I’m not sufficiently proficient to figure out where this happens.
    In case anyone is interested, enclosed are 4 of the profiler summaries.
    Both versions were run on a fast desktop pc with MAMP and PHP 7.
    On a dedicated server running php natively, the timing was similar.

    Attachments:
    You must be logged in to view attached files.
Viewing 1 post (of 1 total)
  • You must be logged in to reply to this topic.