0

I have a Java application running on Spring with a service that creates an invoice, each invoice has 10 read and writes. However we cannot get it to pass the 150TPS (transactions per second), hardware doesnt reach 30% capacity and we have set maxpool size to 300 and DB is set to 3000 i/o, we see no improvements and I'm not sure its code or configuration

AWS DB Monitor

public class InvoiceHomeService {

    private final Logger log = LoggerFactory.getLogger(InvoiceHomeService.class);

    private final DosageService dosageService;

    private final DosageRepository dosageRepository;

    private final InvoiceService invoiceService;

    private final ApplicationProperties applicationProperties;

    public Invoice create(Company company,
                          CreateInvoiceHomeRequest request,
                          Branch branch,
                          EconomicActivity economicActivity,
                          String modalityId) throws DosageNotAvailableException, DateOutRangeException {

        Sorter invoiceType = new Sorter(SortersConstants.HOME);
        DosageTransaction dosageTransaction = getDosageTransaction(request, company, economicActivity, invoiceType, modalityId);

        Invoice invoice = new Invoice();

        /* Critital information */
        invoice.setExternalId(request.getId());
        invoice.setInvoiceType(invoiceType);
        invoice.setInvoiceMethod(new Sorter(SortersConstants.IM_VIRTUAL));
        invoice.setTotalAnticipatedPayment(BigDecimal.ZERO);

        /* Basic information */
        invoice.setContractId(request.getContractId());
//        invoice.setCustomerId(request.getCustomerId());
        invoice.setBillingPeriod(request.getBillingPeriod());
        invoice.setVoucherNumber(request.getVoucherNumber());
        invoice.setVoucherType(request.getVoucherType());
//        invoice.setVoucherSerial(request.getVoucherSerial());
        invoice.setVoucherState(request.getVoucherState());
        invoice.setPaymentType(request.getTypeOfPayment());
        invoice.setEmailNotification(request.getEmailNotification());
        invoice.setPhoneNumberNotification(request.getPhoneNumberNotification());
        invoice.setIsTigo(request.getIsTigo());
        invoice.setPersonType(request.getTypeOfPerson());

        /* Extras information */
//        invoice.setExtraPhoneLine(request.getExtraPhoneLine());
//        invoice.setExtraPlan(request.getExtraPlan());
        invoice.setExtraCustomerAddress(request.getExtraCustomerAddress());
        invoice.setExtraPeriodStartdate(request.getExtraPeriodStartdate());
        invoice.setExtraPeriodEnddate(request.getExtraPeriodEnddate());
        invoice.setExtraPaydayLimit(request.getExtraPaymentDeadline());
        invoice.setExtraServiceInterruptionDate(request.getExtraServiceInterruptionDate());
        invoice.setExtraPeriodDays(request.getExtraPeriodDays());

        /* Customer Information */
        invoice.setNitCustomer(request.getNitCustomer());
        invoice.setSocialReason(request.getSocialReason());

        /* Invoice template */
        invoice.setInvoiceTemplate(JasperSingleton.INVOICE_TEMPLATE_HOME_VERSION);

        Set<InvoiceDetails> details = new HashSet<>();
        for (CreateInvoiceDetailsHomeRequest detail : request.getDetails()) {
            InvoiceDetails item = new InvoiceDetails();
            item.setQuantity(detail.getQuantity());
            item.setConcept(detail.getConcept());
            item.setUnitPrice(detail.getUnitPrice());
            item.setSequence(detail.getSequence());
            item.setSubtotal(detail.getSubtotal());
            item.setExtraPeriod(detail.getExtraPeriod());
            details.add(item);
        }
        invoice.setDetails(details);

        invoice = invoiceService.createInvoice(
            invoice,
            dosageTransaction.getDosage(),
            company,
            branch,
            economicActivity,
            dosageTransaction.getSequence(),
            request.getEmissionDate(),
            request.getEmissionDate(),
            request.getTotalAmount(),
            economicActivity.getType()
        );

        return invoice;
    }

    public synchronized DosageTransaction getDosageTransaction(CreateInvoiceHomeRequest request, Company company, EconomicActivity economicActivity, Sorter invoiceType, String modalityId) throws DosageNotAvailableException, DateOutRangeException {
        Optional<Dosage> optionalDosage = dosageService.getDosageActive(company.getId(), invoiceType.getSequence(),
            economicActivity.getId(), SortersConstants.DT_CYCLE, modalityId, invoiceType.getId());
        if (!optionalDosage.isPresent()) throw new DosageNotAvailableException();
        Dosage dosage = optionalDosage.get();

        if (!request.getEmissionDate().isAfter(dosage.getStartDate().minusDays(1)) ||
            !request.getEmissionDate().isBefore(dosage.getDeadlineDate().plusDays(1))) throw new DateOutRangeException();

        if (modalityId.equalsIgnoreCase(SortersConstants.DM_CICLICAL) && dosage.getSequence() > applicationProperties.getDosageConfig().getMaxInvoicePerDosage().longValue()) {
            dosage.setState(new Sorter(SortersConstants.DS_LOCKED));
            dosage = dosageRepository.save(dosage);
            log.info("Change state dosage: {}, state: {}", dosage.getId(), dosage.getState().getId());
            return getDosageTransaction(request, company, economicActivity, invoiceType, modalityId);
        }

        Long sequence = invoiceService.getSequenceDosageSync(dosage.getId());
        return new DosageTransaction(dosage, sequence);
    }
}
Robert Harvey
  • 178,213
  • 47
  • 333
  • 501
Schopen
  • 1
  • 2
  • So why do you think you need 300 TPS? Is there some specific software requirement that you're trying to meet? The goal isn't necessarily to saturate the CPU utilization; there might be other limiting factors. Wouldn't it be easier to just stand up another AWS instance? – Robert Harvey Feb 28 '21 at 19:01
  • Yes, we need to meet 300 Transactions per second to handle load and clustering is not an option. – Schopen Feb 28 '21 at 19:04
  • Using a sequence will impact performance as it needs to lock the sequence table for every transaction. Is it a cached sequence or maybe call `nextval` inline with the the insert statement? – Bart Feb 28 '21 at 19:05
  • Thanks for replying, sequence is cached – Schopen Feb 28 '21 at 19:06
  • 1
    Well, what sort of profiling tools do you have at your disposal? In any other ordinary application, the first thing I would be doing is *measuring.* – Robert Harvey Feb 28 '21 at 19:08
  • We are using, Jmeter+ Hardware monitoring and Cloudwatch monitors for RDS (postgres). We know that hardware is not the issue, simple 'gets' with 1 database read go up to 1000 TPS – Schopen Feb 28 '21 at 19:32
  • 1
    Why is getDosageTransaction synchronized? What happens, if not? Did you measure you network activity and how long a roundtrip takes? What are the transaction boundaries? – juwil Feb 28 '21 at 20:28

1 Answers1

0

Following up, we found that generating the sequence with JAVA was creating the bottlekneck.

Using "nexval" so postgresql creates the sequence fired our performance to 800TPS

Schopen
  • 1
  • 2