What is causing my program to run out of memory using NodeMCU?

Question

Can someone please tell me why my program runs out of memory? I'm using the Lua 5.1.4 on SDK 1.5.4.1 NodeMCU. Here is my code

wifi.setmode(wifi.STATION)

wifi.sta.config("asdf","xxx")

elWiFi =(wifi.sta.getip())
if elWiFi ~= nil then
    print(wifi.sta.getip())
end

if srv~=nil then
    srv:close()
end

srv=net.createServer(net.TCP) 
if srv ~= nil then
    srv:listen(6969,function(conn) 
        if conn ~= nil then
        conn:on("receive",function(sck,numbers) 
        if numbers ~= nil then 
            collectgarbage("collect")
            p = string.find(numbers, "x=") --parses TCP string
            q = string.find(numbers, "&y")
            if p ~= nill then
                if q ~=  nil then
                    x = (string.sub(numbers,p+2, q-1))
                    print("x=" .. x )       -- prints x value
                end
            end --p ~= nill
            p = string.find(numbers, "y=")
            q = string.find(numbers, "&z")
            if p ~= nil then
                if q ~=  nil then
                    y = (string.sub(numbers,p+2, q-1))
                    print("y=" .. y)          --prints y value
                end
            end --p ~= nill
            p = string.find(numbers, "z=")
            q = string.find(numbers, " H")
            if p ~= nill then
                if q ~=  nil then
                    z = (string.sub(numbers,p+2, q-1))
                    --print("z=" .. z)
                end
            end-- p ~= nill
        end --numbers ~= nil
        conn:close()
        --conn = nil

        print(collectgarbage("count")*1024)
        collectgarbage("collect")
        --print(collectgarbage("count")*1024)

        --conn:send("test\n")

        end)
        end
    end)

end

I print the Heap memory after each it parses x and y before and after collecting garbare. It increases over time and eventually crashes.

Here is what the output looks like

x=-0.003997802734375
y=-0.0095672607421875
6744
6744
x=-0.0029449462890625
y=-0.0099945068359375
7133
7098
.
.
.
x=-0.003662109375
y=-0.00982666015625
35309
35275
x=-0.00311279296875
y=-0.0097503662109375
35424
35389
E:M 64
PANIC: unprotected error in call to Lua API (not enough memory)

 ets Jan  8 2013,rst cause:2, boot mode:(3,7)

load 0x40100000, len 25936, room 16 
tail 0
chksum 0x5b
load 0x3ffe8000, len 2268, room 8 
tail 4
chksum 0xe4
load 0x3ffe88dc, len 8, room 4 
tail 4
chksum 0xd9
csum 0xd9
„ã ì ƒNì’r‚òn|ä d dld` „ãrÛ$Œ ò „  ìdà

NodeMCU custom build by frightanic.com
 branch: master
 commit: 7b83bbb2ea134cd85ac9d63108603cc02c4e20f7
 SSL: false
 modules: adc,file,gpio,net,node,ow,pwm,spi,struct,tmr,uart,websocket,wifi
 build  built on: 2016-11-15 00:43
 powered by Lua 5.1.4 on SDK 1.5.4.1(39cb9a32)
> x=-0.003997802734375

And it restarts after that.

Thanks for the help in advance.

score 2 · Accepted Answer · edited May 23 '17 at 12:13

2

Within the conn:on("receive") callback you mustn't reference the conn variable but the sck variable and here's why:

The callback registered for the 'connection' event keeps a reference to conn and conn keeps reference to the callback closure. This creates a perfect a reference cycle. Because that cannot be garbage collected there's your memory leak.

Furthermore, you shouldn't close the connection within an event callback.

For more discussions see https://github.com/nodemcu/nodemcu-firmware/issues/1303 and https://stackoverflow.com/a/37379426/131929.

edited May 23 '17 at 12:13

Community

1
1

answered Nov 22 '16 at 22:06

Marcel Stör

22,695
19
92
198

It sounds strange. Reference cycles do not prevent Lua from collecting the garbage. – Egor Skriptunoff Nov 22 '16 at 22:25
1

@EgorSkriptunoff, No but the on function registers the CB in the Lua registry and the current net code doesn't clear these down as the object __GC function isn't fired as it persists in the open upval, so the mark and sweep doesn't flush them out. No typical Lua behaviour, but that's what happens. – TerryE Nov 22 '16 at 22:57
@Marcel I'm not sure I follow. I'm not referencing`conn` within `conn:on("receive")` I don't even use `sck` either. Other than the `conn:close()` but even then after i removed that my program hangs after only 3 inputs. – user3249979 Nov 23 '16 at 01:40
Yes you are, you are doing a `conn:close()`. This should be `sck:close()` – TerryE Nov 26 '16 at 19:04
@TerryE It solved my problem. Thank you. I was misunderstanding that i couldn't use `close()` at all. – user3249979 Nov 27 '16 at 00:18
1

@user3249979 You shouldn't close the _connection_. Closing the _socket_ is fine and recommended. [Upvote/accept](http://meta.stackexchange.com/q/5234/257994) the helpful answer(s) so that Stack Overflow can mark this as solved. – Marcel Stör Nov 27 '16 at 10:14

score 1 · Answer 2 · answered Nov 26 '16 at 22:27

This upval leak in net is a bummer. The easiest way to avoid the accidental upval binding is to hoist the declaration of the receiver routine. Something like the following. I've also recoded your parser albeit with the same syntax assumptions in more economic Lua. (Not debugged so I might have a typo, etc.)

local srv=net.createServer(net.TCP)
local function receiver(sck,numbers)
  local p={}
  for v,n in (numbers or ''):gmatch("([x-z])=(%d+)") do
    print (("%s=%s"):format(v,n))
    p[v]=n+0
  end
  -- processReq(p.x, p.y, p.z)
  print(collectgarbage("count")*1024)
  collectgarbage("collect")
  sck:close()
end

if svr then
  srv:listen(6969,function(conn) conn:on("receive",receiver) end)
end

What is causing my program to run out of memory using NodeMCU?

2 Answers2

Linked