1

We were trying out all 6 combinations of ijk model (i.e. IJK, JIK, KIJ, IKJ, JKI, KJI) for matrix multiplication to observe the difference in time taken by each combination. We made a script and ran it overnight on a laptop with Intel i7 9750H and Windows 10 for our observations.

We measured time vs array size. The issue is that the graph we are obtaining from our observations doesn't agree with the theory behind the difference in performance of all combinations i.e. inner loop having to traverse in row manner will have low cache misses and better performance and inner loop having to traverse in column manner will have maximum cache misses and worst performance.

This is the graph from our observations: Our Graph

This is our reference graph: Our Reference

Although the graph has cycles per inner loop iteration vs array size, shouldn't time vs array size be something similar to it? Is our method or code for observation wrong? We are unable to find the fault and can't conclude a result. It would be really great if anyone can guide us on where we are going wrong.

Code we ran for observation:

import time
import numpy as np
import random
import csv

#size = int(input("Enter the size of arrays: "))
# []
sizeArr = np.array([100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700], dtype=int)

for size in sizeArr:
    with open('observation.csv', 'a+') as file:
        writer = csv.writer(file)

        print(f"Size -> {size}\n")
        writer.writerow([f"Size -> {size}"])
        # Initialize for all the sizes
        A = np.zeros((size, size), dtype=int)
        B = np.zeros((size, size), dtype=int)
        C = np.zeros((size, size), dtype=int)

        # Filling arrays with random numbers
        for a in range(size):
            for b in range(size):
                A[a][b] = random.randrange(1, 101, 1)
                B[a][b] = random.randrange(1, 101, 1)

        # ijk
        before = time.time()
        for i in range(size):
            for j in range(size):
                C[i][j] = 0
                for k in range(size):
                    C[i][j] += A[i][k] * B[k][j]
        after = time.time()
        timeTaken = int(after - before)
        print(f"Time taken by ijk -> {timeTaken}")
        writer.writerow([f"Time taken by ijk -> {timeTaken}"])

        # Reset C elements to 0
        C = np.zeros((size, size), dtype=int)

        # Filling arrays with random numbers
        for a in range(size):
            for b in range(size):
                A[a][b] = random.randrange(1, 101, 1)
                B[a][b] = random.randrange(1, 101, 1)

        # jik
        before = time.time()
        for j in range(size):
            for i in range(size):
                C[i][j] = 0
                for k in range(size):
                    C[i][j] += A[i][k] * B[k][j]
        after = time.time()
        timeTaken = int(after - before)
        print(f"Time taken by jik -> {timeTaken}")
        writer.writerow([f"Time taken by jik -> {timeTaken}"])

        # Reset C elements to 0
        C = np.zeros((size, size), dtype=int)

        # Filling arrays with random numbers
        for a in range(size):
            for b in range(size):
                A[a][b] = random.randrange(1, 101, 1)
                B[a][b] = random.randrange(1, 101, 1)

        # kij
        before = time.time()
        for k in range(size):
            for i in range(size):
                temp = A[i][k]
                for j in range(size):
                    C[i][j] += temp * B[k][j]
        after = time.time()
        timeTaken = int(after - before)
        print(f"Time taken by kij -> {timeTaken}")
        writer.writerow([f"Time taken by kij -> {timeTaken}"])

        # Reset C elements to 0
        C = np.zeros((size, size), dtype=int)

        # Filling arrays with random numbers
        for a in range(size):
            for b in range(size):
                A[a][b] = random.randrange(1, 101, 1)
                B[a][b] = random.randrange(1, 101, 1)

        # ikj
        before = time.time()
        for i in range(size):
            for k in range(size):
                temp = A[i][k]
                for j in range(size):
                    C[i][j] += temp * B[k][j]
        after = time.time()
        timeTaken = int(after - before)
        print(f"Time taken by ikj -> {timeTaken}")
        writer.writerow([f"Time taken by ikj -> {timeTaken}"])

        # Reset C elements to 0
        C = np.zeros((size, size), dtype=int)

        # Filling arrays with random numbers
        for a in range(size):
            for b in range(size):
                A[a][b] = random.randrange(1, 101, 1)
                B[a][b] = random.randrange(1, 101, 1)

        # jki
        before = time.time()
        for j in range(size):
            for k in range(size):
                temp = B[k][j]
                for i in range(size):
                    C[i][j] += A[i][k] * temp
        after = time.time()
        timeTaken = int(after - before)
        print(f"Time taken by jki -> {timeTaken}")
        writer.writerow([f"Time taken by jki -> {timeTaken}"])

        # Reset C elements to 0
        C = np.zeros((size, size), dtype=int)

        # Filling arrays with random numbers
        for a in range(size):
            for b in range(size):
                A[a][b] = random.randrange(1, 101, 1)
                B[a][b] = random.randrange(1, 101, 1)

        # kji
        before = time.time()
        for k in range(size):
            for j in range(size):
                temp = B[k][j]
                for i in range(size):
                    C[i][j] += A[i][k] * temp
        after = time.time()
        timeTaken = int(after - before)
        print(f"Time taken by kji -> {timeTaken}")
        writer.writerow([f"Time taken by kji -> {timeTaken}"])
        print("\n")

rpanai
  • 12,515
  • 2
  • 42
  • 64
  • On windows you need something like `perf stat` on Linux - harware counters access . Perhaps this would help: https://stackoverflow.com/questions/34641644/is-there-a-windows-equivalent-of-the-linux-command-perf-stat – Severin Pappadeux Jan 10 '20 at 15:28
  • You should 1/ divide the time by n^3 to get a time/iteration. 2/ you cannot have reliable time estimates with a unique run of a function. Run your function 10 times and keep the smallest result. 3/ there is an overhead when doing benchmarks with python. You should use C and the highest optimization level. – Alain Merigot Jan 10 '20 at 22:16

0 Answers0